Displaying a device within an endoluminal image stack

ABSTRACT

Apparatus and methods are described including, while an endoluminal data-acquisition device is being moved through a subject&#39;s lumen, acquiring a plurality of endoluminal data points of the lumen using the endoluminal data-acquisition device. It is determined that respective endoluminal data points correspond to respective locations along the lumen. A display is driven to display at least some of the plurality of endoluminal data points in a stack. While a second endoluminal device is inside the lumen, a current location of at least a portion of the second endoluminal device with respect to the lumen is determined. In response thereto, an image of the second endoluminal device is displayed within the stack, at a location within the stack corresponding to the current location of the second endoluminal device. Other applications are also described.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of PCT Application no.PCT/IL2013/050438 to Steinberg (published as WO 13/175,472), filed May21, 2013, which:

(i) claims the benefit of:

-   -   U.S. Provisional Patent Application 61/688,730, filed May 21,        2012; and    -   U.S. Provisional Patent Application 61/761,709, filed Feb. 7,        2013;

(ii) is a continuation-in-part of U.S. Ser. No. 13/228,229 to Tolkowsky(published as US 2012/0004537), filed Sep. 8, 2011, which is acontinuation of International Application No. PCT/IL2011/000612 toTolkowsky (published as WO 12/014,212), filed 28 Jul. 2011, which claimsthe benefit of:

-   -   U.S. Provisional Patent Application 61/344,464, filed 29 Jul.        2010;    -   U.S. Provisional Patent Application 61/344,875, filed 1 Nov.        2010;    -   U.S. Provisional Patent Application 61/457,339, filed 3 Mar.        2011;    -   U.S. Provisional Patent Application 61/457,455, filed 1 Apr.        2011;    -   U.S. Provisional Patent Application 61/457,780, filed 2 Jun.        2011; and    -   U.S. Provisional Patent Application 61/457,951, filed 15 Jul.        2011; and

(iii) is a continuation-in-part of U.S. patent application Ser. No.12/666,879 to Steinberg (published as US 2012/0230565), which is the USnational phase of PCT Application No. PCT/IL2009/001089 to Cohen(published as WO 10/058,398), filed Nov. 18, 2009, which claims thebenefit of:

-   -   U.S. Provisional Patent Application 61/193,329, filed Nov. 18,        2008;    -   U.S. Provisional Patent Application 61/193,915, filed Jan. 8,        2009;    -   U.S. Provisional Patent Application 61/202,181, filed Feb. 4,        2009;    -   U.S. Provisional Patent Application 61/202,451, filed Mar. 2,        2009;    -   U.S. Provisional Patent Application 61/213,216, filed May 18,        2009;    -   U.S. Provisional Patent Application 61/213,534, filed Jun. 17,        2009;    -   U.S. Provisional Patent Application 61/272,210, filed Sep. 1,        2009; and    -   U.S. Provisional Patent Application 61/272,356, filed Sep. 16,        2009.

The present application is related to the following patent applications:

-   -   U.S. patent application Ser. No. 12/075,214 to Iddan (published        as 2008/0221439), filed Mar. 10, 2008, entitled “Tools for use        with moving organs.”    -   U.S. patent application Ser. No. 12/075,252 to Iddan (published        as US 2008/0221440), filed Mar. 10, 2008, entitled “Imaging and        tools for use with moving organs.”    -   U.S. patent application Ser. No. 12/075,244 to Tolkowsky        (published as US 2008/0221442), filed Mar. 10, 2008, entitled        “Imaging for use with moving organs.”    -   U.S. patent application Ser. No. 12/781,260 to Blank (published        as US 2010/0228076), filed May 17, 2010, entitled “Controlled        actuation and deployment of a medical device.”    -   U.S. patent application Ser. No. 12/487,315 to Iddan (published        as US 2009/0306547), filed Jun. 18, 2009, entitled “Stepwise        advancement of a medical tool,” which claims the benefit of U.S.        Provisional Patent Application No. 61/129,331 to Iddan, filed on        Jun. 19, 2008, entitled “Stepwise advancement of a medical        tool.”

All of the above-mentioned applications are incorporated herein byreference.

FIELD OF EMBODIMENTS OF THE INVENTION

Some applications of the present invention generally relate to medicalimaging. Specifically, some applications of the present invention relateto the co-use of endoluminal data and extraluminal imaging.

BACKGROUND

Vascular catheterizations, such as coronary catheterizations, arefrequently-performed medical interventions. Such interventions aretypically performed in order to diagnose the blood vessels for potentialdisease, and/or to treat diseased blood vessels. Typically, in order toenable observation of blood vessels, the catheterization is performedunder extraluminal imaging. Additionally, for some procedures, anendoluminal data-acquisition device is used to perform endoluminalimaging and/or measurements. The extraluminal imaging and, whereapplicable, the endoluminal data are typically evaluated by the medicalstaff in combination with one another in the course of the intervention,as well as post procedurally.

SUMMARY OF EMBODIMENTS

Some applications of the present invention are applied to medicalprocedures performed, in whole or in part, on or within luminalstructures. For some applications, apparatus and methods are providedfor facilitating the co-use of extraluminal imaging and endoluminal data(i.e., data that are acquired using an endoluminal data-acquisitiondevice), in performing medical procedures. Endoluminal data may includeimaging data (e.g., imaging data acquired using an endoluminal imagingprobe), data derived from measurements (e.g., measurements performedusing an endoluminal sensor or measuring device), other data, and anycombination thereof.

There is therefore provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminal deviceconfigured to be moved through a lumen of a subject's body, one or moreextraluminal imaging devices configured to acquire extraluminal imagesof the lumen, and a display, the apparatus including:

at least one processor configured to receive from the one or moreextraluminal imaging devices:

-   -   a first set of extraluminal images of the lumen, the lumen being        visible in at least some of the first set of images, and    -   a second set of extraluminal images of the endoluminal device        inside the lumen, the second set of extraluminal images being        acquired while the endoluminal device is being moved through the        lumen; the at least one processor including:    -   roadmap-image-designation functionality configured to designate        at least one of the first set of images as a roadmap image;    -   pathway-designation functionality configured to designate,        within the lumen in the roadmap image, a roadmap pathway;    -   feature-identifying functionality configured for at least a        portion of the images belonging to the second set of        extraluminal images to identify within the image a plurality of        the features that are visible within the image;    -   roadmap-mapping functionality configured for the at least a        portion of the images belonging to the second set of        extraluminal images:        -   to compare an arrangement of three or more of the features            within the image to a shape of at least a portion of the            roadmap pathway, and        -   based upon the comparing, to map the identified features to            locations along the roadmap pathway within the roadmap            image; and    -   output-generation functionality configured, in response to the        mapping of the identified features to locations along the        roadmap pathway within the roadmap image, to generate an output        on the display.

For some applications, the roadmap-mapping functionality is configuredto compare the arrangement of three or more features within the image tothe shape of the portion of the roadmap pathway by comparing vectorsdefined by pairs of the features within the image to vectors defined bypairs of locations on the roadmap pathway.

For some applications, the roadmap-mapping functionality is configuredto compare the arrangement of three or more features within the image tothe shape of the portion of the roadmap pathway by comparing an angledefined by vectors defined by pairs of the features within the image toan angle defined by the roadmap pathway.

For some applications, the roadmap-mapping functionality is configuredto compare the arrangement of three or more features within the image tothe shape of the portion of the roadmap pathway by comparing distancesbetween by pairs of the features within the image to the shape of atleast the portion of the roadmap pathway.

For some applications, the at least one processor further includespathway-calibration functionality configured, based upon the mapping, todetermine a plurality of local calibration factors associated withrespective portions of the roadmap image.

For some applications,

the endoluminal device includes a first endoluminal data-acquisitiondevice configured to acquire a plurality of endoluminal data pointswhile the endoluminal data-acquisition device is being moved through thelumen from a starting location within the lumen,

the at least one processor further includes co-registrationfunctionality configured to co-register respective endoluminal datapoints to respective locations within the roadmap image, by:

-   -   identifying the starting location of the endoluminal        data-acquisition device in the roadmap image, and    -   determining a distance from the starting location at which        respective endoluminal data points were acquired, based upon the        speed at which the endoluminal data-acquisition device was moved        through the lumen, the frame rate at which the endoluminal data        points were acquired, and the local calibration factor        associated with the respective portions of the roadmap image;        and

the output-generation functionality is configured to generate an outputon the display based upon the co-registration of the endoluminal datapoints to the respective locations within the roadmap image.

For some applications,

the apparatus is for use with a second endoluminal data-acquisitiondevice configured to acquire an additional plurality of endoluminal datapoints while the second endoluminal data-acquisition device is beingmoved through the lumen, and

the co-registration functionality is configured to co-registerrespective endoluminal data points of the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective endoluminal data points of theplurality of endoluminal data points acquired by the first endoluminaldata-acquisition device by co-registering the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective locations within the roadmapimage.

For some applications, the endoluminal data-acquisition device includesan endoluminal optical coherence tomography device configured to acquireoptical coherence tomography images, and the co-registrationfunctionality is configured to co-register respective endoluminal datapoints to respective locations within the roadmap image byco-registering respective optical coherence tomography images torespective locations within the roadmap image.

For some applications, the at least one processor is configured, basedupon the mapping, to determine locations of the endoluminal device inrespective extraluminal images of the second set of extraluminal imageswith respect to the roadmap image.

For some applications, the at least one processor is configured, on-linewith respect to acquisitions of the extraluminal images of the secondset of extraluminal images, to determine locations of the endoluminaldevice in respective extraluminal images of the second set ofextraluminal images with respect to the roadmap image, and theoutput-generation functionality is configured to generate an output thatis indicative of the determined on-line location of the endoluminaldevice with respect to the roadmap image.

For some applications,

the endoluminal device includes a first endoluminal data-acquisitiondevice configured to acquire a plurality of endoluminal data pointswhile the endoluminal data-acquisition device is being moved through thelumen,

the at least one processor further includes co-registrationfunctionality that is configured, based upon determining locations ofthe endoluminal device in respective extraluminal images of the secondset of extraluminal images with respect to the roadmap image, toco-register respective endoluminal data points to respective locationswithin the roadmap image, and

the output-generation functionality is configured to generate an outputon the display based upon the co-registration of the endoluminal datapoints to the respective locations within the roadmap image.

For some applications,

the apparatus is for use with a second endoluminal data-acquisitiondevice configured to acquire an additional plurality of endoluminal datapoints while the second endoluminal data-acquisition device is beingmoved through the lumen, and

the co-registration functionality is configured to co-registerrespective endoluminal data points of the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective endoluminal data points of theplurality of endoluminal data points acquired by the first endoluminaldata-acquisition device by co-registering the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective locations within the roadmapimage.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the co-registration functionality isconfigured to co-register respective endoluminal data points torespective locations within the roadmap image by co-registeringrespective endoluminal images to respective locations within the roadmapimage.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and theco-registration functionality is configured to co-register respectiveendoluminal data points to respective locations within the roadmap imageby co-registering respective functional endoluminal data points torespective locations within the roadmap image.

For some applications, the at least one processor further includesstack-generation functionality that is configured, based upon theco-registration, to generate a stack of endoluminal data points, inwhich relative dispositions of endoluminal data points within the stackcorrespond to relative locations of the endoluminal data points withrespect to the roadmap image.

For some applications,

the at least one processor further includes parameter-measurementfunctionality that is configured, based upon the co-registering of theendoluminal data points to respective locations within the roadmapimage, to determine a parameter of a portion of the lumen thatcorresponds to a portion of the stack of endoluminal data points, and

the output-generation functionality is configured to generate the outputin response to the determined parameter.

For some applications, the parameter-measurement functionality isconfigured to determine a length of a portion of the lumen thatcorresponds to a portion of the stack of endoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal deviceconfigured to be moved through a lumen of a subject's body, anextraluminal imaging device configured to acquire extraluminal images ofthe lumen, and a display, the method including:

using the extraluminal imaging device, acquiring a first set ofextraluminal images of the lumen, the lumen being visible in at leastsome of the first set of images;

designating at least one of the first set of images as a roadmap image;

designating, within the lumen in the roadmap image, a roadmap pathway;

moving the endoluminal device through at least a portion of the lumen;

while the endoluminal device is being moved through the lumen, acquiringa second set of extraluminal images of the endoluminal device inside thelumen, using the extraluminal imaging device;

for at least a portion of the images belonging to the second set ofextraluminal images:

-   -   identifying within the image a plurality of the features that        are visible within the image;    -   comparing an arrangement of three or more of the features within        the image to a shape of at least a portion of the roadmap        pathway; and    -   based upon the comparing, mapping the identified features to        locations along the roadmap pathway within the roadmap image;        and in response thereto, generating an output on the display.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminal deviceconfigured to be moved through a lumen of a subject's body, one or moreextraluminal imaging devices configured to acquire extraluminal imagesof the lumen, and a display, the apparatus including:

at least one processor configured to receive from the one or moreextraluminal imaging devices:

-   -   a first set of extraluminal images of the lumen, the lumen being        visible in at least some of the first set of images, and    -   a second set of extraluminal images of the endoluminal device        inside the lumen, the second set of extraluminal images being        acquired while the endoluminal device is being moved through the        lumen; the at least one processor including:    -   roadmap-image-designation functionality configured to designate        at least one of the first set of images as a roadmap image;    -   feature-identifying functionality configured for at least a        portion of the images belonging to the second set of        extraluminal images to identify within the image a plurality of        the features that are visible within the image;    -   pathway-calibration functionality configured, in response to the        identified features in the images belonging to the second set of        extraluminal images, to determine a plurality of local        calibration factors associated with respective portions of the        roadmap image; and    -   output-generation functionality configured, based upon one or        more of the determined local calibration factors, to generate an        output on the display.

For some applications, the pathway-calibration functionality isconfigured to determine the plurality of local calibration factorsassociated with respective portions of the roadmap image based upon aknown speed at which the endoluminal device is moved through the lumen.

For some applications, the pathway-calibration functionality isconfigured to determine the plurality of local calibration factorsassociated with respective portions of the roadmap image by determininglocal relative calibration factors of the portions of the roadmap imagewith respect to each other.

For some applications, the pathway-calibration functionality isconfigured to determine the plurality of local calibration factorsassociated with respective portions of the roadmap image based upon aknown physical dimension associated with one or more of the identifiedfeatures.

For some applications, the pathway-calibration functionality isconfigured to determine the plurality of local calibration factorsassociated with respective portions of the roadmap image based upon aknown physical distance between two or more of the identified features.

For some applications, the pathway-calibration functionality isconfigured to determine the plurality of local calibration factorsassociated with respective portions of the roadmap image based upon aknown physical dimension of one or more of the identified features.

For some applications,

the at least one processor further includes:

-   -   pathway-designation functionality configured to designate,        within the lumen in the roadmap image, a roadmap pathway; and    -   roadmap-mapping functionality configured for the at least a        portion of the images belonging to the second set of        extraluminal images:        -   to compare an arrangement of three or more of the features            within the image to a shape of at least a portion of the            roadmap pathway, and        -   based upon the comparing, to map the identified features to            locations along the roadmap pathway within the roadmap            image; and

the pathway-calibration functionality is configured to determine theplurality of local calibration factors associated with respectiveportions of the roadmap image based upon the mapping.

For some applications, the pathway-calibration functionality isconfigured to determine the plurality of local calibration factorsassociated with respective portions of the roadmap image by determininga plurality of local calibration factors associated with respectiveportions of the roadmap pathway.

For some applications,

the endoluminal device includes a first endoluminal data-acquisitiondevice configured to acquire a plurality of endoluminal data pointswhile the endoluminal data-acquisition device is being moved through thelumen from a starting location within the lumen; and

the at least one processor further includes co-registrationfunctionality configured to co-register respective endoluminal datapoints to respective locations within the roadmap image, by:

-   -   identifying the starting location of the endoluminal        data-acquisition device in the roadmap image; and    -   determining a distance from the starting location at which        respective endoluminal data points were acquired, based upon the        speed at which the endoluminal data-acquisition device was moved        through the lumen, the frame rate at which the endoluminal data        points were acquired, and the local calibration factor        associated with the respective portions of the roadmap image;        and

the output-generation functionality is configured to generate an outputon the display based upon the co-registration of the endoluminal datapoints to the respective locations within the roadmap image.

For some applications,

the apparatus is for use with a second endoluminal data-acquisitiondevice configured to acquire an additional plurality of endoluminal datapoints while the second endoluminal data-acquisition device is beingmoved through the lumen, and

the co-registration functionality is configured to co-registerrespective endoluminal data points of the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective endoluminal data points of theplurality of endoluminal data points acquired by the first endoluminaldata-acquisition device by co-registering the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective locations within the roadmapimage.

For some applications, the endoluminal data-acquisition device includesan endoluminal optical coherence tomography device configured to acquireoptical coherence tomography images, and the co-registrationfunctionality is configured to co-register respective endoluminal datapoints to respective locations within the roadmap image byco-registering respective optical coherence tomography images torespective locations within the roadmap image.

For some applications, the at least one processor is configured, basedupon the local calibration factors, to determine locations of theendoluminal device in respective extraluminal images of the second setof extraluminal images with respect to the roadmap image.

For some applications, the at least one processor is configured, on-linewith respect to acquisitions of the extraluminal images of the secondset of extraluminal images, to determine locations of the endoluminaldevice in respective extraluminal images of the second set ofextraluminal images with respect to the roadmap image, and theoutput-generation functionality is configured to generate an output thatis indicative of the determined on-line location of the endoluminaldevice with respect to the roadmap image.

For some applications,

the endoluminal device includes a first endoluminal data-acquisitiondevice configured to acquire a plurality of endoluminal data pointswhile the endoluminal data-acquisition device is being moved through thelumen,

the at least one processor further includes co-registrationfunctionality that is configured, based upon determining locations ofthe endoluminal device in respective extraluminal images of the secondset of extraluminal images with respect to the roadmap image, toco-register respective endoluminal data points to respective locationswithin the roadmap image, and

the output-generation functionality is configured to generate an outputon the display based upon the co-registration of the endoluminal datapoints to the respective locations within the roadmap image.

For some applications,

the apparatus is for use with a second endoluminal data-acquisitiondevice configured to acquire an additional plurality of endoluminal datapoints while the second endoluminal data-acquisition device is beingmoved through the lumen, and

the co-registration functionality is configured to co-registerrespective endoluminal data points of the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective endoluminal data points of theplurality of endoluminal data points acquired by the first endoluminaldata-acquisition device by co-registering the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective locations within the roadmapimage.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the co-registration functionality isconfigured to co-register respective endoluminal data points torespective locations within the roadmap image by co-registeringrespective endoluminal images to respective locations within the roadmapimage.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and theco-registration functionality is configured to co-register respectiveendoluminal data points to respective locations within the roadmap imageby co-registering respective functional endoluminal data points torespective locations within the roadmap image.

For some applications, the at least one processor further includesstack-generation functionality that is configured, based upon theco-registration, to generate a stack of endoluminal data points, inwhich relative dispositions of endoluminal data points within the stackcorrespond to relative locations of the endoluminal data points withrespect to the roadmap image.

For some applications,

the at least one processor further includes parameter-measurementfunctionality that is configured, based upon the co-registering of theendoluminal data points to respective locations within the roadmapimage, to determine a parameter of a portion of the lumen thatcorresponds to a portion of the stack of endoluminal data points, and

the output-generation functionality is configured to generate the outputin response to the determined parameter.

For some applications, the parameter-measurement functionality isconfigured to determine a length of a portion of the lumen thatcorresponds to a portion of the stack of endoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal deviceconfigured to be moved through a lumen of a subject's body, anextraluminal imaging device configured to acquire extraluminal images ofthe lumen, and a display, the method including:

using the extraluminal imaging device, acquiring a first set ofextraluminal images of the lumen, the lumen being visible in at leastsome of the first set of images;

designating at least one of the first set of images as a roadmap image;

moving the endoluminal device through the lumen;

while the endoluminal device is being moved through the lumen, acquiringa second set of extraluminal images of the endoluminal device inside thelumen, using the extraluminal imaging device;

identifying, within each of at least a portion of the images belongingto the second set of extraluminal images, a plurality of features thatare visible in the image;

in response to the identified features in the images belonging to thesecond set of extraluminal images, determining a plurality of localcalibration factors associated with respective portions of the roadmapimage; and

generating an output on the display, based upon one or more of thedetermined local calibration factors.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminal device theendoluminal device having at least one radiopaque portion associatedtherewith and being configured to be moved through a lumen of a subject,one or more extraluminal imaging devices configured to acquireextraluminal images of the lumen, and a display, the apparatusincluding:

a reference tool, the reference tool having coupled thereto radiopaquemarkers, a characteristic of the markers varying along a least a portionof the reference tool, the reference tool being configured to beinserted into the lumen; and

at least one processor configured:

-   -   while the endoluminal device is being moved through the lumen,        to operate the extraluminal imaging device to acquire a        plurality of extraluminal images of the lumen; and    -   to determine that, at times corresponding to the acquisitions of        respective extraluminal images of the lumen, the endoluminal        device was at respective locations within the lumen, by        determining, within the extraluminal images of the lumen,        locations of the at least one radiopaque portion associated with        the endoluminal device with respect to the radiopaque markers of        the reference tool;

the processor including output-generation functionality configured togenerate an output on the display in response to the determinedlocations of the endoluminal device within the lumen.

For some applications, the at least one processor is configured, on-linewith respect to acquisitions of the extraluminal images of the lumen, todetermine locations of the endoluminal device with respect to the lumen,the output-generation functionality being configured to generate theoutput by generating an output that is indicative of the determinedon-line location of the endoluminal device with respect to the lumen.

For some applications, a distance between pairs of markers that arecoupled to the reference tool varies along at least the portion of thereference tool.

For some applications, a shape of the markers that are coupled to thereference tool varies along at least the portion of the reference tool.

For some applications, a pattern of the markers that are coupled to thereference tool varies along at least the portion of the reference tool.

For some applications, the reference tool includes a guide toolconfigured to guide the movement of the endoluminal device within thelumen.

For some applications, the guide tool includes a tool selected from thegroup consisting of: a sheath, and a wire.

For some applications,

the endoluminal device includes a first endoluminal data-acquisitiondevice configured to acquire a plurality of endoluminal data pointswhile the endoluminal data-acquisition device is being moved through thelumen,

the at least one processor includes co-registration functionalityconfigures, based on determining that at times corresponding to theacquisitions of respective extraluminal images of the lumen theendoluminal device was at respective locations within the lumen, toco-register respective endoluminal data points to respective locationsalong the lumen, and

the output-generation functionality is configured to generate the outputby generating an output on the display based upon the co-registration ofthe endoluminal data points to the respective locations along the lumen.

For some applications,

the apparatus is for use with a second endoluminal data-acquisitiondevice configured to acquire an additional plurality of endoluminal datapoints while the second endoluminal data-acquisition device is beingmoved through the lumen, and

the co-registration functionality is configured to co-registerrespective endoluminal data points of the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective endoluminal data points of theplurality of endoluminal data points acquired by the first endoluminaldata-acquisition device by co-registering the additional plurality ofendoluminal data points acquired by the second endoluminaldata-acquisition device to respective locations along the lumen.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the co-registration functionality isconfigured to co-register respective endoluminal data points torespective locations along the lumen by co-registering respectiveendoluminal images to respective locations along the lumen.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and theco-registration functionality is configured to co-register respectiveendoluminal data points to respective locations along the lumen byco-registering respective functional endoluminal data points torespective locations along the lumen.

For some applications, the at least one processor further includesstack-generation functionality that is configured, based upon theco-registration, to generate a stack of endoluminal data points, inwhich relative dispositions of endoluminal data points within the stackcorrespond to relative locations of the endoluminal data points withrespect to the lumen.

For some applications,

the at least one processor further includes parameter-measurementfunctionality that is configured, based upon the co-registering of theendoluminal data points to respective locations along the lumen, todetermine a parameter of a portion of the lumen that corresponds to aportion of the stack of endoluminal data points, and

the output-generation functionality is configured to generate the outputin response to the determined parameter.

For some applications, the parameter-measurement functionality isconfigured to determine a length of a portion of the lumen thatcorresponds to a portion of the stack of endoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal device theendoluminal device being configured to be moved through a lumen of asubject and having at least one radiopaque portion associated therewith,an extraluminal imaging device configured to acquire extraluminal imagesof the lumen, and a display, the method including:

providing a reference tool, the reference tool having coupled theretoradiopaque markers, a characteristic of the markers varying along aleast a portion of the reference tool;

inserting the reference tool into the lumen;

moving the endoluminal device through the lumen;

while the endoluminal device is being moved through the lumen, operatingthe extraluminal imaging device to acquire a plurality of extraluminalimages of the lumen;

operating at least one processor to determine that, at timescorresponding to the acquisitions of respective extraluminal images ofthe lumen, the endoluminal device was at respective locations within thelumen, by determining, within the extraluminal images of the lumen,locations of the at least one radiopaque portion associated with theendoluminal device with respect to the radiopaque markers of thereference tool; and

operating the processor to generate an output on the display in responsethereto.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject, and a display,the apparatus including:

at least one processor configured to determine that, at at least onelocation, an event occurred, the event being selected from the groupconsisting of: two or more endoluminal data points having been acquired,and no endoluminal data point having been acquired;

the at least one processor including:

-   -   stack-generation functionality configured to generate a stack of        the endoluminal data points, in which the endoluminal data        points are positioned at locations corresponding to relative        locations within the lumen at which the endoluminal data points        were acquired, and in which the event is accounted for; and    -   display-driving functionality configured to drive the display to        display the stack.

For some applications, the display-driving functionality is configuredto drive the display to display a length scale in relation to thedisplayed stack of the endoluminal data points.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the stack-generation functionality isconfigured to generate the stack by generating an endoluminal imagestack.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and thestack-generation functionality is configured to generate the stack bygenerating a stack of functional endoluminal data points.

For some applications, the stack-generation functionality is configuredto generate the stack of endoluminal data points by generating a stackof indications of the endoluminal data points, locations of theindications within the stack corresponding to relative locations withinthe lumen at which the endoluminal data points were acquired.

For some applications, the at least one processor is configured todetermine that at the at least one location the event occurred bydetermining that at the at least one location two or more endoluminaldata points were acquired.

For some applications, the stack-generation functionality is configuredto account for the event by including in the stack only one of theendoluminal data points that was acquired at the location.

For some applications, the at least one processor is configured todetermine that at the at least one location the event occurred bydetermining that at the at least one location no endoluminal data pointwas acquired.

For some applications, the stack-generation functionality is configuredto account for the event by including in the stack a gap at a locationwithin the stack that corresponds to the location within the lumen atwhich no endoluminal data point was acquired.

For some applications, the at least one processor further includesparameter-measurement functionality configured to measure a parameter ofa portion of the lumen, based upon the stack of the endoluminal datapoints.

For some applications, the parameter-measurement functionality isconfigured to measure a length of the portion of the lumen, based uponthe stack of the endoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal data-acquisitiondevice configured to acquire endoluminal data points while movingthrough a lumen of a subject, including:

while the endoluminal data-acquisition device is being moved through thelumen, acquiring a plurality of endoluminal data points of the lumenusing the endoluminal data-acquisition device;

determining that, at at least one location, an event occurred, the eventbeing selected from the group consisting of: two or more endoluminaldata points having been acquired, and no endoluminal data point havingbeen acquired; and

displaying the endoluminal data points in a stack, in which theendoluminal data points are positioned at locations corresponding torelative locations within the lumen at which the endoluminal data pointswere acquired, and in which the event is accounted for.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject, and a display,the apparatus including:

at least one processor configured to determine that, at at least onelocation, an event occurred, the event being selected from the groupconsisting of: two or more endoluminal data points having been acquired,and no endoluminal data point having been acquired;

the at least one processor including:

-   -   stack-generation functionality configured to generate a stack of        the endoluminal data points, in which the endoluminal data        points are positioned at locations corresponding to relative        locations within the lumen at which the endoluminal data points        were acquired, and in which the event is accounted for;    -   parameter-measurement functionality configured to measure a        parameter of a portion of the lumen, based upon the stack of the        endoluminal data points; and    -   output-generation functionality configured to generate an output        on the display based upon the measured length.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the stack-generation functionality isconfigured to generate the stack by generating an endoluminal imagestack.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and thestack-generation functionality is configured to generate the stack bygenerating a stack of functional endoluminal data points.

For some applications, the stack-generation functionality is configuredto generate the stack of endoluminal data points by displaying a stackof indications of the endoluminal data points, locations of theindications within the stack corresponding to relative locations withinthe lumen at which the endoluminal data points were acquired.

For some applications, the at least one processor is configured todetermine that at the at least one location the event occurred bydetermining that at the at least one location two or more endoluminaldata points were acquired.

For some applications, the stack-generation functionality is configuredto account for the event by including in the stack only one of theendoluminal data points that was acquired at the location.

For some applications, the at least one processor is configured todetermine that at the at least one location the event occurred bydetermining that at the at least one location no endoluminal data pointwas acquired.

For some applications, the stack-generation functionality is configuredto account for the event by including in the stack a gap at a locationwithin the stack that corresponds to the location within the lumen atwhich no endoluminal data point was acquired.

For some applications, the parameter-measurement functionality isconfigured to measure a length of the portion of the lumen, based uponthe stack of the endoluminal data points.

For some applications, the output-generation functionality is configuredto drive the display to display the stack of endoluminal data points andto display a length scale in relation to the displayed stack of theendoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal data-acquisitiondevice configured to acquire endoluminal data points while movingthrough a lumen of a subject, including:

while the endoluminal data-acquisition device is being moved through thelumen, acquiring a plurality of endoluminal data points of the lumenusing the endoluminal data-acquisition device;

determining that, at at least one location, an event occurred, the eventbeing selected from the group consisting of: two or more endoluminaldata points having been acquired, and no endoluminal data point havingbeen acquired;

displaying the endoluminal data points in a stack, in which theendoluminal data points are positioned at locations corresponding torelative locations within the lumen at which the endoluminal data pointswere acquired, and in which the event is accounted for; and

determining a parameter of a portion of the lumen based upon thedisplayed stack of endoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject, and a display,the apparatus including:

at least one processor including:

-   -   stack-generation functionality configured to generate a stack of        the endoluminal data points;    -   co-registration functionality configured to co-register the        endoluminal data points to respective locations along the lumen        in an extraluminal image of the lumen;    -   parameter-measurement functionality configured, based upon the        co-registering of the endoluminal data points to respective        locations along the lumen in the extraluminal image of the        lumen, to determine a parameter of a portion of the lumen that        corresponds to a portion of the stack of endoluminal data        points; and    -   output-generation functionality configured to generate an output        on the display, in response to the measured parameter.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the stack-generation functionality isconfigured to generate the stack by generating an endoluminal imagestack.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and thestack-generation functionality is configured to generate the stack bygenerating a stack of functional endoluminal data points.

For some applications, the stack-generation functionality is configuredto generate the stack of endoluminal data points by displaying a stackof indications of the endoluminal data points, locations of theindications within the stack corresponding to relative locations withinthe lumen at which the endoluminal data points were acquired.

For some applications, the stack-generation functionality is configuredto include in the stack a gap at a location within the stack thatcorresponds to a location within the lumen at which no endoluminal datapoint was acquired.

For some applications, the stack-generation functionality is configuredto not include within the stack at least one endoluminal data point thatwas acquired at a location along the lumen at which another endoluminaldata point was acquired.

For some applications, the parameter-measurement functionality isconfigured to determine the parameter of the portion of the lumen thatcorresponds to the portion of the stack of endoluminal data points bydetermining a length of the portion of the lumen that corresponds to theportion of the stack.

For some applications, the output-generation functionality is configuredto drive the display to display the stack of endoluminal data points andto display a length scale in relation to the displayed stack of theendoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal data-acquisitiondevice configured to acquire endoluminal data points while movingthrough a lumen of a subject, including:

while the endoluminal data-acquisition device is being moved through thelumen, acquiring a plurality of endoluminal data points of the lumenusing the endoluminal data-acquisition device;

displaying the endoluminal data points in a stack;

co-registering the endoluminal data points to respective locations alongthe lumen in an extraluminal image of the lumen;

based upon the co-registering of the endoluminal data points torespective locations along the lumen in the extraluminal image of thelumen, determining a parameter of a portion of the lumen thatcorresponds to a portion of the stack of endoluminal data points; and

generating an output in response thereto.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject, and a display,the apparatus including:

at least one processor configured to determine that, at at least onelocation, no endoluminal data point was acquired;

the at least one processor including output-generation functionalityconfigured to generate an output on the display using at least a portionof the plurality of endoluminal data points of the lumen acquired usingthe endoluminal data-acquisition device, the output including anindication that no endoluminal data point was acquired at the location.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the output-generation functionality isconfigured to generate the output using a plurality of acquiredendoluminal images of the lumen.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen, while the endoluminaldata-acquisition device is being moved through the lumen, and theoutput-generation functionality is configured to generate the outputusing a plurality of acquired functional endoluminal data pointsregarding the lumen.

For some applications,

the at least one processor includes stack-generation functionalityconfigured to generate a stack of the endoluminal data points, in whichthe endoluminal data points are positioned at locations corresponding torelative locations within the lumen at which the endoluminal data pointswere acquired, the stack including a gap in the stack at a locationwithin the stack that corresponds to the location within the lumen atwhich no endoluminal data point was acquired; and

the output-generation functionality is configured to generate the outputby driving the display to display the stack of endoluminal data points.

For some applications, the at least one processor further includesparameter-measurement functionality configured to measure a length of aportion of the lumen, based upon the stack of the endoluminal datapoints.

For some applications, the output-generation functionality is configuredto drive the display to display a length scale in relation to thedisplayed stack of the endoluminal data points.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal data-acquisitiondevice configured to acquire endoluminal data points while movingthrough a lumen of a subject's body, including:

while the endoluminal data-acquisition device is being moved through thelumen, acquiring a plurality of endoluminal data points of the lumenusing the endoluminal data-acquisition device;

determining that, at at least one location, no endoluminal data pointwas acquired;

generating an output using at least a portion of the plurality ofendoluminal data points of the lumen acquired using the endoluminaldata-acquisition device, the output including an indication that noendoluminal data point was acquired at the location.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject's body, and adisplay, the apparatus including:

at least one processor including:

-   -   stack-generation functionality configured to:        -   determine that the endoluminal data points are not aligned            with each other due to non-longitudinal motion undergone by            the endoluminal data-acquisition device with respect to the            lumen, between acquisitions of respective endoluminal data            points; and        -   in response thereto, align the endoluminal data points with            each other, to at least partially account for the            non-longitudinal motion undergone by the endoluminal            data-acquisition device; and    -   output generation functionality configured to generate an output        on the display based upon the aligned endoluminal data points.

For some applications, the stack-generation functionality is configuredto determine that the endoluminal data points are not aligned with eachother by determining that the endoluminal data points are not alignedwith each other due to a portion of the endoluminal data-acquisitiondevice having rotated about a longitudinal axis of the endoluminaldata-acquisition device, between acquisitions of respective endoluminaldata points.

For some applications, the stack-generation functionality is configuredto determine that the endoluminal data points are not aligned with eachother by determining that the endoluminal data points are not alignedwith each other due to a portion of the endoluminal data-acquisitiondevice having become tilted, between acquisitions of respectiveendoluminal data points.

For some applications, the stack-generation functionality is configuredto determine that the endoluminal data points are not aligned with eachother by determining that the endoluminal data points are not alignedwith each other due to a portion of the endoluminal data-acquisitiondevice having moved axially, between acquisitions of respectiveendoluminal data points.

For some applications, the apparatus further includes a sensor coupledto a portion of the data-acquisition device and configured to detect anon-longitudinal orientation of the portion of the data-acquisitiondevice, and the stack-generation functionality is configured todetermine that the endoluminal data points are not aligned with eachother by detecting the non-longitudinal orientation of the portion ofthe data-acquisition device via the sensor.

For some applications, the stack-generation functionality is configuredto align the endoluminal data points with each other by aligning theendoluminal data points with each other using image processing.

For some applications, the endoluminal data points include endoluminalimages, and the stack-generation functionality is configured to alignthe endoluminal data points with each other by:

identifying a region of one of the endoluminal images as having a givencharacteristic;

identifying a region in an adjacent endoluminal image that has the samecharacteristic; and

aligning the adjacent images with one another by aligning the regions ofeach of the images.

For some applications,

the at least one processor is further configured to receive a pluralityof extraluminal images of the lumen while the endoluminaldata-acquisition device is being moved through the lumen,

the endoluminal data-acquisition device includes at least a portionthereof that is visible in the extraluminal images, and

the stack-generation functionality is configured to determine that theendoluminal data points are not aligned with each other, by determininga disposition of the endoluminal data-acquisition device with respect tothe lumen at times at which respective extraluminal images wereacquired, by performing image processing on the extraluminal images.

For some applications, the visible portion of the endoluminaldata-acquisition device includes a portion that is asymmetric withrespect to a longitudinal axis of the endoluminal data-acquisitiondevice, and the stack-generation functionality is configured todetermine the disposition of the endoluminal data-acquisition devicewith respect to the lumen at times at which respective extraluminalimages were acquired by analyzing an appearance of the asymmetricportion in the respective extraluminal images.

For some applications, the stack-generation functionality is configuredto align the endoluminal data points with each other by determining acenterline of the lumen and aligning the endoluminal data points withrespect to the centerline.

For some applications, the stack-generation functionality is configuredto determine the centerline of the lumen by determining a straightenedcenterline of the lumen, and the stack-generation functionality isconfigured to align the endoluminal data points with respect to thecenterline by aligning the endoluminal data points with respect to thestraightened centerline.

For some applications, the stack-generation functionality is configuredto generate a stack of the endoluminal data points, based upon thealignment of the endoluminal data points, and the output generationfunctionality is configured to generate the output on the display bygenerating a display of the stack of endoluminal data points on thedisplay.

For some applications, the endoluminal data-acquisition device includesan endoluminal imaging device that is configured to acquire a pluralityof endoluminal images while the endoluminal imaging device is beingmoved through the lumen, and the stack-generation functionality isconfigured to generate the stack by generating an endoluminal imagestack.

For some applications, the endoluminal data-acquisition device includesan endoluminal data-acquisition device that is configured to acquirefunctional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and thestack-generation functionality is configured to generate the stack bygenerating a stack of functional endoluminal data points.

For some applications, the stack-generation functionality is configuredto generate the stack by generating a stack of indications of theendoluminal data points, locations of the indications within the stackcorresponding to relative locations within the lumen at which theendoluminal data points were acquired.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal data-acquisitiondevice configured to acquire endoluminal data points while movingthrough a lumen of a subject's body, and a display, the methodincluding:

while the endoluminal data-acquisition device is being moved through thelumen, acquiring a plurality of endoluminal data points of the lumenusing the endoluminal data-acquisition device;

determining that the endoluminal data points are not aligned with eachother due to non-longitudinal motion undergone by the endoluminaldata-acquisition device with respect to the lumen, between acquisitionsof respective endoluminal data points;

in response thereto, aligning the endoluminal data points with eachother, to at least partially account for the non-longitudinal motionundergone by the endoluminal data-acquisition device; and

generating an output on the display based upon the aligned endoluminaldata points.

There is further provided, in accordance with some applications of thepresent invention, apparatus for use with an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject's body, a secondendoluminal device configured to be moved through the lumen, and adisplay, the apparatus including:

at least one processor configured to:

determine that respective endoluminal data points correspond torespective locations along the lumen, and

to determine a current location of the second endoluminal device withrespect to the lumen;

the at least one processor including:

-   -   stack-generation functionality configured to generate a stack of        the endoluminal data points, in which the endoluminal data        points are positioned at locations corresponding to relative        locations within the lumen at which the endoluminal data points        were acquired; and    -   display-driving functionality configured to drive the display to        display the stack, and to display within the stack an image of        the second endoluminal device at a location within the stack        corresponding to the current location of the second endoluminal        device.

For some applications, the display-driving functionality is configuredto drive the display to display the image of the second endoluminaldevice within the stack by driving the display to display a virtualrepresentation of the second endoluminal device within the stack.

For some applications, the display-driving functionality is configuredto drive the display to display the image of the second endoluminaldevice within the stack by driving the display to display a real imageof the second endoluminal device within the stack.

There is further provided, in accordance with some applications of thepresent invention, a method for use with an endoluminal data-acquisitiondevice configured to acquire endoluminal data points while movingthrough a lumen of a subject's body, and a display, the methodincluding:

while the endoluminal data-acquisition device is being moved through thelumen, acquiring a plurality of endoluminal data points of the lumenusing the endoluminal data-acquisition device;

determining that respective endoluminal data points correspond torespective locations along the lumen;

driving the display to display at least some of the plurality ofendoluminal data points in a stack;

while a second endoluminal device is inside the lumen, determining acurrent location of at least a portion of the second endoluminal devicewith respect to the lumen; and

in response thereto, displaying within the stack an image of the secondendoluminal device at a location within the stack corresponding to thecurrent location of the second endoluminal device.

The present invention will be more fully understood from the followingdetailed description of embodiments thereof, taken together with thedrawings, in which:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a flow chart, at least some of the steps of which are used inprocedures that utilize co-use of endoluminal data and extraluminalimaging, in accordance with some applications of the present invention;

FIG. 1B is a block diagram of an endoluminal data-acquisition device, anextraluminal image acquisition device, a user interface, a display, anda processor that are used, in accordance with some applications of thepresent invention;

FIGS. 2A-E are schematic illustrations of images of a lumen of asubject, in accordance with some applications of the present invention;

FIG. 3A is a graph indicating a typical type of movement of anendoluminal data-acquisition device during pullback of the device;

FIG. 3B is a graph indicating another typical type of movement of anendoluminal data-acquisition device during pullback of the device;

FIG. 3C is a schematic illustration of an endoluminal image stack thatincludes gaps therein, in accordance with some applications of thepresent invention;

FIG. 4 shows the co-use of previously-acquired endoluminal images and anextraluminal fluoroscopic image, in accordance with some applications ofthe present invention; and

FIG. 5 is a schematic illustration of a reference tool having markerscoupled thereto, a characteristic of the markers varying along thelength of at least a portion of the reference tool, in accordance withsome applications of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS

The terms “medical tool,” “tool”, “device,” and “probe” refer to anytype of a diagnostic or therapeutic or other functional tool including,but not limited to, a cardiovascular catheter, a stent delivery,placement and/or retrieval tool, a balloon delivery and/or placementand/or retrieval tool, a valve delivery and/or repair and/or placementand/or retrieval tool, a graft delivery and/or placement and/orretrieval tool, a tool for the delivery and/or placement and/orretrieval of an implantable device or of parts of such device, animplantable device or parts thereof, a tool for closing a gap, a toolfor closing a septal defect, a guide wire, a marker wire, a suturingtool, a clipping tool (such as a valve-leaflet-clipping tool), a biopsytool, an aspiration tool, a navigational tool, a localization tool, aprobe comprising one or more location sensors, a tissue characterizationprobe, a probe for the analysis of fluid, a measurement probe, anelectrophysiological probe, a stimulation probe, an ablation tool, atool for penetrating or opening partial or total occlusions in bloodvessels, a drug or substance delivery tool, a chemotherapy tool, aphotodynamic therapy tool, a brachytherapy tool, a local irradiationtool, a laser device, a tool for delivering energy, a tool fordelivering markers or biomarkers, a tool for delivering biological glue,an irrigation device, a suction device, a ventilation device, a devicefor delivering and/or placing and/or retrieving a lead of anelectrophysiological device, a lead of an electrophysiological device, apacing device, a coronary sinus device, an imaging device, a sensingprobe, a probe comprising an optical fiber, a robotic tool, a tool thatis controlled remotely, an excision tool, a plaque excision tool (suchas a plaque excision catheter), or any combination thereof.

The terms “image” and “imaging” refer to any type of medical images orimaging, typically resulting in the generation of a sequence of imagesand including, but not limited to, imaging using ionizing radiation,imaging using non-ionizing radiation, video, fluoroscopy, angiography,ultrasound, CT, MR, PET, PET-CT, CT angiography, SPECT, Gamma cameraimaging, Optical Coherence Tomography (OCT), Near-Infra-Red Spectroscopy(NIRS), Vibration Response Imaging (VRI), optical imaging, infraredimaging, electrical mapping imaging, other forms of functional imaging,Focused Acoustic Computed Tomography (FACT), Optical Frequency DomainImaging (OFDI), or any combination or fusion thereof. Examples ofultrasound imaging include Endo-Bronchial Ultrasound (EBUS),Trans-Thoracic Echo (TTE), Trans-Esophageal Echo (TEE), Intra-VascularUltrasound (IVUS), Intra-Cardiac Ultrasound (ICE), or any combinationthereof.

The term “contrast agent,” when used in reference to its application inconjunction with imaging, refers to any substance that is used tohighlight, and/or enhance in another manner, the anatomical structure,functioning, and/or composition of a bodily organ while the organ isbeing imaged.

The term “stabilized,” when used in the context of displayed images,means a display of a series of images in a manner such that periodic,cyclical, and/or other motion of the body organ(s) being imaged, and/orof a medical tool being observed, is partially or fully reduced, withrespect to the entire image frame, or at least a portion thereof.

The term “automatic,” when used for describing the generation andutilization of the roadmap, means “without necessitating userintervention or interaction.” (Such interaction or intervention maystill however be optional in some cases.)

The term “real-time” means without a noticeable delay.

The term “near real-time” means with a short noticeable delay (such asapproximately one or two motion cycles of the applicable organ, and, inthe case of procedures relating to organs or vessels the motion of whichare primarily a result of the cardiac cycle, less than two seconds).

The term “on-line,” when used in reference to image processing, or tomeasurements being made on images, means that the image processing isperformed, and/or the measurements are made, intra-procedurally, inreal-time or near real-time.

Applications of the present invention are typically used during medicalprocedures that are performed, in whole or in part, on or within luminalstructures. For some applications, apparatus and methods provided hereinfacilitate the co-use of extraluminal imaging and endoluminal data inperforming such medical procedures. Endoluminal data may include imagingdata, data derived from measurements, other data, or any combinationthereof.

For some applications, the co-use of the endoluminal data and theextraluminal images is performed in the following manner. Endoluminaldata are acquired by positioning an endoluminal data-acquisition devicealong a luminal segment of interest that includes a designated luminalsite. Subsequently, while observing extraluminal images of the luminalsegment, one or more locations along that segment are indicated by auser input device. In response to the indication of the one or morelocations by the user input device, the corresponding,previously-acquired endoluminal images are displayed.

Typically, the designated luminal site includes a site being diagnosed,at which, subject to the outcome of the diagnosis, a therapeutic devicewill be positioned and deployed, e.g., the site of an anatomicalfeature, the implantation site of a previously-implanted device, and/ora site at a defined location with respect to the implantation site. Forexample, the designated luminal site may include a portion of the lumenthat is narrow with respect to surrounding portions of the lumen, and/orthe site of a lesion.

For some applications, the co-use of the endoluminal data and theextraluminal images is performed in the following manner. Endoluminaldata are acquired by positioning an endoluminal data-acquisition deviceat a designated luminal site. Subsequently, an endoluminal therapeuticdevice is positioned and deployed at the designated luminal site underextraluminal imaging, while concurrently viewing on-line the endoluminaldata that were previously acquired by the endoluminal data-acquisitiondevice at the current location of the therapeutic device. Typically,endoluminal data are acquired at respective endoluminal sites in thevicinity of the designated endoluminal site. Subsequently, when theendoluminal therapeutic device is placed inside the lumen,previously-acquired endoluminal data are displayed and updated,typically automatically and typically on-line, to correspond to thecurrent location of the therapeutic device (or of a portion thereof),the location of the therapeutic device typically changing during thepositioning of the therapeutic device.

For some applications, extraluminal imaging and the previously-acquiredendoluminal data are co-used such that it is as if the therapeuticdevice is being positioned and deployed under both real-timeextraluminal imaging and real-time endoluminal data acquisition. This isbecause (a) the extraluminal imaging is performed in real-time, and (b),although the endoluminal data are not acquired in real-time, endoluminaldata are displayed that correspond to the current location of thetherapeutic device.

In accordance with some applications of the present invention, when thetherapeutic device is disposed inside the lumen, the location of thedevice within the lumen is determined by performing image processing onthe extraluminal image of the device inside the lumen.

For some applications, the image processing includes tracking of one ormore visible portions of a moving therapy-applying portion of the devicein the extraluminal images. Typically, the tracking is performed inreal-time, and, typically, in accordance with techniques described in US2010/0228076 to Blank, which is incorporated herein by reference.

For some applications, the image processing includes stabilization of animage stream produced by the extraluminal imaging. Typically, thestabilization is performed in real-time, and typically in accordancewith techniques described in US 2008/0221442 to Tolkowsky, or US2010/0228076 to Blank, both of which applications are incorporatedherein by reference. Typically, the stabilization facilitates the co-useof the endoluminal data with the extraluminal images (particularly incases of intense organ motion). This is because it is typically easierto determine the luminal location of the therapeutic device based upon astabilized image stream than to determine the luminal location of thetherapeutic device on a native, non-stabilized image stream.

For some applications, the stabilized image stream is also enhanced,typically in real-time, typically in accordance with techniquesdescribed in US 2010/0228076 to Blank.

For some applications, during the acquisition of the endoluminal data bythe endoluminal data-acquisition device, the location of the endoluminaldata-acquisition device is determined by moving the endoluminaldata-acquisition device under extraluminal imaging and image processingthe extraluminal images to determine the location of a movingdata-acquiring portion of the endoluminal data-acquisition device. Forsome applications, during this stage, the extraluminal image stream isstabilized and/or enhanced, as described hereinabove, to facilitate thedetermination of the location of the endoluminal data-acquisitiondevice, based upon the extraluminal images. Alternatively, othertechniques are used for determining the location of the endoluminaldata-acquisition device, as described hereinbelow.

For some applications, the luminal structure to which the apparatus andmethods described herein are applied includes a lumen in the vascularsystem, the respiratory tract, the digestive tract, the urinary tract,or any other luminal structure within a patient's body.

For some applications, the endoluminal data-acquisition device is animaging probe. For some applications, the imaging probe is an IVUSprobe, an EBUS probe, another ultrasound probe, an OCT probe, an NIRSprobe, an MR probe, a FACT probe, an OFDI probe, or any combinationthereof.

For some applications, the endoluminal data-acquisition device performsadditional functions. For example, the endoluminal data-acquisitiondevice may comprise a probe, such as the VIBE(™) RX Vascular ImagingBalloon Catheter, marketed by Volcano Corporation (San Diego, USA), thatincludes both IVUS and coronary balloon functionalities.

For some applications, the endoluminal data-acquisition device acquiresdata in a form other than images. For example, the data may include datarelated to pressure, flow, temperature, electrical activity,oxygenation, biochemical composition, or any combination thereof. Forsome applications, and typically when data are acquired with respect toa coronary vessel, the endoluminal data-acquisition device is aFractional Flow Reserve (FFR) probe, and/or an instantaneous wave-freeratio (iFR) probe. For some applications, FFR and/or iFR measurementsare determined by performing image-processing on extraluminal images,and the derived FFR and/or iFR measurements are co-registered withendoluminal images of the lumen, using techniques described herein. Forsome applications, FFR and/or iFR measurements are determined byperforming image-processing on endoluminal images, and the derived FFRand/or iFR measurements are co-registered with extraluminal images ofthe lumen, using techniques described herein. For some applications,endoluminal images are co-registered with extraluminal images of thelumen, using techniques described herein, and FFR and/or iFRmeasurements are determined by performing image-processing on theco-registered images.

For some applications, the extraluminal imaging is fluoroscopy, CT, MR,PET, SPECT, ultrasound, or any combination thereof.

For some applications, the apparatus and methods described herein areused with a therapeutic device that is positioned and/or deployed at ananatomical feature that requires or potentially requires treatment, suchas a partial or total occlusion, a native valve, an aneurism, adissection, a malformation, a septal defect, a mass suspected of beingmalignant, a mass suspected of being inflammatory, etc. The endoluminaldata are typically acquired at, and/or in the vicinity of, theanatomical feature.

For some applications, apparatus and methods described herein are usedwith a therapeutic device that is positioned and/or deployed at animplantation site of a previously-implanted device such as a stent, agraft or a replacement valve. The endoluminal data are determined at,and/or in the vicinity of, the implantation site. For example, thetechniques described herein may be used during the placement of a newprosthetic aortic valve at the site of (e.g., inside) a previouslyimplanted prosthetic aortic valve that is no longer functioning.

For some applications, apparatus and methods described herein are usedwith a therapeutic device that is positioned and/or deployed at adefined location relative to a previously-implanted device such as astent, a graft or a replacement valve. The endoluminal data aredetermined at and in the vicinity of the defined location. For example,the techniques described herein may be used during the placement of acoronary stent such that the new stent overlaps with or is adjacent to apreviously-implanted stent, in order to treat a long lesion and/or alesion that has diffused along a coronary artery.

Reference is now made to FIG. 1A, which is a flow chart, at least someof the steps of which are used in the course of co-use of endoluminaldata and extraluminal imaging, in accordance with some applications ofthe current invention. It is noted that, for some applications, some ofthe steps shown in FIG. 1A may be practiced, without all of the stepsshown in FIG. 1A necessarily being practiced in combination.

Reference is also made to FIG. 1B, which is a block diagram of anendoluminal data-acquisition device 16, an extraluminal imageacquisition device 17, a user interface 18, a display 19, and aprocessor 20. Processor 20 is typically used to perform the proceduredescribed with respect to FIG. 1A. Processor 20 typically receivesinputs via the image acquisition device and the user interface, andgenerates an output on display 19. For some applications, the userinterface includes a keyboard, a mouse, a trackball, a joystick, atouchscreen monitor, a touchpad, a voice-command interface, and/or othertypes of user interfaces that are known in the art. Typically, thedisplay includes a monitor. For some applications, the display includesa head-up display and/or a head-mounted display, such as Google Glass.Processor 20 typically includes at least some of the followingfunctionalities, the functions of which are described in further detailhereinbelow: roadmap-image-designation functionality 21,pathway-designation functionality 22, landmark-classificationfunctionality 23, feature-identifying functionality 24, roadmap-mappingfunctionality 25, location-interpolation functionality 26,pathway-calibration functionality 27, co-registration functionality 28,stack-generation functionality 29, parameter-measurement functionality30, duplicate-data-point-identification functionality 31,data-point-selection functionality 32, display-driving functionality 33,direction-determination functionality 34, output-generationfunctionality 35, and/or region-identification functionality 36. It isnoted that, for some applications, processor 20 does not include all ofthe above-listed functionalities, but rather includes only some of theabove-listed functionalities. It is further noted that, for someapplications, more than one processor is used to perform theabove-listed functionalities, or a portion thereof. For someapplications, more than one extraluminal imaging device is used withprocessor 20. For example, a first extraluminal imaging device may beused to acquire a first set of extraluminal images (e.g., as describedhereinbelow with reference to phase 1 of FIG. 1A), and a secondextraluminal imaging device may be used to acquire a second set ofextraluminal images (e.g., as described hereinbelow with reference tophase 5 of FIG. 1A).

In phase 1, a first set of extraluminal images is acquired, in which thelumen is visible. Typically, an angiographic image sequence is acquired,while there is contrast agent inside the lumen.

In phase 2, roadmap-image-designation functionality 21 of processor 20selects an image from the first set of extraluminal images, anddesignates the selected image as the roadmap image. For someapplications, the image is selected from the first set of extraluminalimages manually by a user. Alternatively, the image is selectedautomatically. For some applications, a roadmap image is automaticallyselected by processor 20, but the processor allows a user to overridethe automatically-selected roadmap image, by manually designating aroadmap image.

For some applications, the automatic selection of an image frame isperformed using techniques described in US 2012/0230565, WO 10/058,398,WO 12/014,212, and/or US 2012/0004537, all of which applications areincorporated herein by reference. For example, the image may be selectedbased upon the following criteria: (a) the image is acquired at adesired cardiac phase (typically end diastole) and (b) in the image, thecontrast agent highlights the lumen. For procedures in which thetechniques described herein are performed on a subject's coronaryarteries, an image may be selected from the set of images based uponvisibility of at least a portion of the coronary arteries in the set ofimages. For some applications, the angiogram with the greatestvisibility of coronary arteries is selected, with such selectiontypically being automatic. The greatest visibility is typicallydetermined based upon the greatest total number of arteries observed,the greatest number of image pixels attributed to an artery, and/or thegreatest image contrast in the appearance of specific arteries. For someapplications, an extraluminal image that is based upon a plurality ofextraluminal images (e.g., an image that is based upon averaging aplurality of images) is selected and designated as the roadmap image.

Reference is now made to FIG. 2A, which shows an image of a subject'sarteries that has been designated as a roadmap image, in accordance withsome applications of the present invention. It may be observed that inthe roadmap image, an artery 40, through which an endoluminaldata-acquisition device will be inserted, is visible.

Referring again to FIG. 1A, in phase 3, pathway-designationfunctionality 22 of processor 20 designates a roadmap pathway within thelumen in the roadmap image.

Reference is now made to FIG. 2B, which shows the roadmap image of FIG.2A, with a roadmap pathway 42 having been designated within artery 40.It is noted that although, in FIG. 2B, path 42 is displayed within theroadmap image, for some applications, the roadmap pathway is designatedwithout the path actually being displayed on a display. Path-designationfunctionality 22 designates the roadmap pathway in response to a manualuser input, and/or automatically. For example, the path may bedesignated by the user indicating some points along the path and theprocessor completing the path, based upon the manually-indicated points.For some applications, the roadmap pathway is at least partiallydetermined by determining the centerlines of the lumen. For example, thecenterlines of the lumen may be determined using techniques fordetermining the centerline of a lumen described in US 2012/0230565, WO10/058,398, WO 12/014,212, and/or US 2012/0004537, all of whichapplications are incorporated herein by reference.

Typically, the roadmap pathway includes at least a portion of the lumenthrough which the endoluminal data-acquisition device will be moved.Further typically, the roadmap pathway is designated in such a manner asto facilitate mapping to the pathway a plurality of features that aretypically visible in extraluminal images of the lumen that are acquiredduring the movement of the endoluminal data-acquisition device throughthe lumen, as described in further detail hereinbelow with reference tophase 7 of the procedure. For some applications, such features includefeatures associated with the endoluminal data-acquisition device such asa data-acquiring portion of the endoluminal data-acquisition device(e.g., the endoluminal data-acquisition device head), radiopaque markersthat are disposed at a fixed location with respect to the data-acquiringportion of the endoluminal data-acquisition device (e.g., endoluminaldata-acquisition device head), a guiding catheter through which theendoluminal data-acquisition device is inserted, the distal end of theguiding catheter, a catheter through which the data-acquiring portion ofthe endoluminal data-acquisition device is moved and/or a portionthereof, and/or a guidewire over which the endoluminal data-acquisitiondevice is inserted and/or a portion thereof, etc. For some applications,such features include anatomical features, such as bifurcations,lesions, calcifications, etc. Alternatively or additionally, suchfeatures include previously-implanted medical devices, such as a stent,or a valve. Such features may be disposed within the lumen through whichthe endoluminal data-acquisition device is moved, or in a portion of thesubject's body in the vicinity of the lumen, e.g., in a lumen thatbranches from the lumen through which the endoluminal data-acquisitiondevice is inserted. For applications in which some of the features aredisposed in a portion of the subject's body in the vicinity of thelumen, the roadmap pathway in the roadmap image typically extends to theportion of the subject body, even if the portion of the subject's bodyis not within the lumen. In accordance with respective applications, theroadmap pathway may be shaped as a curve, a polygon, a branching set oflines and/or curves, and/or another shape.

For some applications, processor 20 includes landmark-classificationfunctionality 23. The landmark-classification functionality classifiesregions within the roadmap image as corresponding to locations withinthe roadmap image within which given features are likely to be. For someapplications, such features include features associated with theendoluminal device such as a data-acquiring portion of the endoluminaldata-acquisition device (e.g., the endoluminal data-acquisition devicehead), radiopaque markers that are disposed at a fixed location withrespect to the data-acquiring portion of the endoluminaldata-acquisition device (e.g., the endoluminal data-acquisition devicehead), a guiding catheter through which the endoluminal data-acquisitiondevice is inserted, the distal end of the guiding catheter, a catheterthrough which the data-acquiring portion of the endoluminaldata-acquisition device is moved and/or a portion thereof, and/or aguidewire over which the endoluminal data-acquisition device is insertedand/or a portion thereof, etc. For some applications, such featuresinclude anatomical features, such as bifurcations, lesions,calcifications, etc. Alternatively or additionally, such featuresinclude previously-implanted medical devices, such as a stent, or avalve. Such features may be disposed within the lumen through which theendoluminal data-acquisition device is moved, or in a portion of thesubject's body in the vicinity of the lumen, e.g., in a lumen thatbranches from the lumen through which the endoluminal data-acquisitiondevice is inserted.

For some applications, the landmark-classification functionalityclassifies landmarks in response to a manual input from a user.Alternatively or additionally, the landmark-classification functionalityclassifies landmarks automatically. For example, thelandmark-classification functionality may analyze the angiographicsequence from which the roadmap was generated. In some of the frames ofthe angiographic sequence, a portion of the above-described features maybe visible, and in other frames of the angiographic sequence, portionsof the lumen may be visible. Thus, the landmark-classificationfunctionality may determine where respective features are with respectto the vessel, and, in response thereto, may classify regions within theroadmap image as corresponding to locations within the roadmap imagewithin which respective features are likely to be. Alternatively oradditionally, the landmark-classification functionality may classifyregions within the roadmap image as corresponding to locations withinthe roadmap image within which the above-described features are likelyto be, by analyzing extraluminal images that are acquired subsequent tothe generation of the roadmap (e.g., extraluminal images that areacquired in phase 5 of the procedure).

Referring again to FIG. 1A, in phase 4, the endoluminal data-acquisitiondevice is inserted toward a designated site. For some applications, thesite is a site being diagnosed, at which, subject to the outcome of suchdiagnosis, a therapeutic device will be positioned and deployed, e.g.,the site of an anatomical feature, the implantation site of apreviously-implanted device, and/or a site at a defined location withrespect to the implantation site, as described hereinabove.

In phase 5, a plurality of endoluminal data points (e.g., images), areacquired by the endoluminal data-acquisition device, while theendoluminal data-acquisition device is being moved through the lumen. Atthe same time, while the endoluminal data-acquisition device is beingmoved through the lumen, a second set of extraluminal images areacquired of the endoluminal data-acquisition device within the lumen.Typically, the second set of extraluminal images are acquired whilethere is an absence of contrast agent within the lumen. For example, aset of fluoroscopic images of the lumen may be acquired. Alternatively,the second set of extraluminal images are acquired in the presence ofcontrast agent in the lumen.

It is noted that, in general, the scope of the present applicationincludes performing the techniques described herein with an endoluminaldata-acquisition device that acquires data points while thedata-acquisition device is being advanced distally through the lumen,and/or an endoluminal data-acquisition device that acquires data pointswhile the data-acquisition device is being refracted proximally throughthe lumen. It is further noted that, in general, the scope of thepresent application includes performing the techniques described hereinwith an endoluminal data-acquisition device that acquires images of thelumen and/or a data-acquisition device that acquires functional dataregarding the lumen.

Typically, data are acquired at and/or in the vicinity of the designatedsite. Typically, a plurality of data points (e.g., images) are acquiredat respective locations along the lumen. It is noted that, for someapplications, data are acquired subsequent to the initial insertion ofthe data-acquisition device into the lumen. For example, when data areacquired from blood vessels, the data-acquisition device is typicallyinserted into the blood vessel to beyond the site of interest underextraluminal imaging (e.g., fluoroscopy), and data acquisition isperformed during (manual or automated) pullback of the data-acquisitiondevice through the blood vessel. In alternative applications, e.g., whendata are acquired from an endobronchial airway, data are typicallyacquired by the data-acquisition device during insertion of thedata-acquisition device into the airway.

For some applications, the commencement and/or termination of pullbackare identified, typically automatically and typically on-line, by meansof image processing. For some applications, the image processing isperformed by an image comparator which identifies a change (such as inthe color of image pixels or in the geometry of image features) in thesequentially-acquired endoluminal images, and interprets the change asindicating the commencement of pullback. For some applications, theimage processing is performed by an image comparator which identifies adiminishing change in the sequentially-acquired endoluminal images, andinterprets the diminishing change as indicating the termination ofpullback.

For some applications, the commencement and/or termination of pullbackare identified by means of a signal transmitted by the pullback unitand/or by the endoluminal data-acquisition system. For someapplications, the commencement and/or termination of pullback areindicated by means of user input.

In phase 6, feature-identifying functionality 24 of processor 20identifies, within at least a portion of the images belonging to thesecond set of extraluminal images, a plurality of features that arevisible within the images. The feature-identifying functionalityclassifies the features as potentially being a given type of feature.For some applications, such feature types include features associatedwith the endoluminal device such as a data-acquiring portion of theendoluminal data-acquisition device (e.g., the endoluminaldata-acquisition device head), radiopaque markers that are disposed at afixed location with respect to the data-acquiring portion of theendoluminal data-acquisition device (e.g., the endoluminaldata-acquisition device head), a guiding catheter through which theendoluminal data-acquisition device is inserted, the distal end of theguiding catheter, a catheter through which the data-acquiring portion ofthe endoluminal data-acquisition device is moved and/or a portionthereof, and/or a guidewire over which the endoluminal data-acquisitiondevice is inserted and/or a portion thereof, etc. For some applications,such features include anatomical features, such as bifurcations,lesions, calcifications, etc. Alternatively or additionally, suchfeatures include previously-implanted medical devices, such as a stent,or a valve. Such features may be disposed within the lumen through whichthe endoluminal data-acquisition device is moved, or in a portion of thesubject's body in the vicinity of the lumen, e.g., in a lumen thatbranches from the lumen through which the endoluminal data-acquisitiondevice is inserted.

For some applications, features are identified in accordance withtechniques described in US 2012/0230565, WO 10/058,398, WO 12/014,212,and/or US 2012/0004537, all of which applications are incorporatedherein by reference. For some applications, feature-identifyingfunctionality 24 of processor 20 uses one or more of the followingtechniques to identify and/or classify the above-described featureswithin the images belonging to the second set of extraluminal images:

-   -   a. Identifying features using image processing techniques (e.g.,        detecting vesselness, using a hessian filter, using corner        detection, using directional filters, etc.)    -   b. Optionally, radiopaque markers can be detected using        techniques described in US 2012/0230565 and/or in WO 10/058,398,        both of which applications are incorporated herein by reference.        For example, the automatic identification of markers may include        some or all of the following phases, which are typically        performed in real-time:        -   1. Pre-processing: Individual image frames (or a region of            interest (ROI) within such frames) are pre-processed in            order to facilitate the subsequent identification of            markers. Such pre-processing typically comprises the            reduction of static and/or dynamic noise, background            removal, or a combination thereof. For some applications, a            median filter, a Mexican hat filter, a directional Mexican            hat filter, and/or a low-pass filter is applied to the            individual image frames. For some applications, the            preprocessing includes the detection and removal from the            image frames of CABG wires, wires and/or electrodes of            implanted tools such as pacemakers or defibrillators, and/or            wires and/or electrodes of external devices such as an ECG            monitor, and/or an external defibrillator.        -   2. Filtering of non-marker-like features: Individual image            frames (or a region of interest within such frames) are            processed to filter out remaining features that are clearly            not markers. For some applications, the filtering includes            the application to the image frames of a median filter, a            Mexican hat filter, a directional Mexican hat filter, a            maximal stable external regions (MSER) filter, an MSER-like            filter, a Hessian filter, or a combination thereof.        -   3. For some applications, Hessian eigenvalues are calculated            for each pixel in each image frame, or for all pixels within            an ROI of the image frame. Typically, local clusters of            pixels with high minimal eigenvalues represent a            “paraboloid-like” area in the image and are identified as            potential radiopaque markers.        -   4. Scoring: Remaining features in individual image frames            (or a region of interest within such frames) are assigned a            “fitness” score (i.e., a “markerness” score, or a “dotness”            score in the case of the most common markers), describing            the likelihood that they are markers. For some applications,            the score is calculated from the abovementioned filtering.        -   5. Matching: Remaining features in individual image frames            (or a region of interest within such frames) are analyzed            for matching with one another. For example, in the case of            aiming to detect the two radiopaque markers of a coronary            balloon, pair matching is performed. Such matching is            typically performed based upon relative location, distance,            orientation, visual similarity, and/or other factors.        -   6. Detection: For some applications, once a pair of clusters            (the clusters within the set being strong candidates to be            tool markers) has been identified as being at a similar            distance from one another and/or relative angle to one            another in several consecutive image frames, the pair of            clusters is determined to be the markers.    -   c. Optionally, the guiding catheter, the guidewire, and        elongated objects such as the endoluminal data-acquisition        device head, can be detected using techniques described in US        2012/0230565, and/or in WO 10/058,398, both of which        applications are incorporated herein by reference. For example,        the image frame may be analyzed such that the extent to which a        given pixel is likely to be an element of an image of an object        in applicable areas of the image frame is determined. For        example, this may be determined by means of a filter, such as        the filter described in an article by Frangi et al., entitled        “Multiscale vessel enhancement filtering” (Medical Image        Computing and Computer Assisted Intervention—MICCAI 1998—Lecture        Notes in Computer Science, vol. 1496, Springer Verlag, Berlin,        Germany, pp. 130-137), which is incorporated herein by        reference, by means of a filter that performs enhancement,        and/or by detection and/or segmentation of curvilinear        structures. For some applications, a filter is used that is        similar to the filter described by Frangi, but that differs from        the filter described by Frangi (a) in that a homogeneous        function is used, and/or (b) in the multipliers employed for the        normalization of scales.    -   d. Classifying features using machine learning techniques. For        example, one or more of the following machine learning        techniques may be used: Support Vector Machine (SVM), Deep        Believe Networks, Neural Networks, and/or Random Decision        Forest.

Reference is now made to FIGS. 2C and 2D, which show, respectively, anexample of a raw fluoroscopic image frame, and the fluoroscopic imageframe with a plurality of features identified and classified therein, inaccordance with some applications of the present invention. As indicatedby the shapes of the features shown in FIG. 2D, the feature-identifyingfunctionality of the processor typically classifies the identifiedfeatures as potentially being a given type of feature. For example, thefeatures indicated by stars 44 are classified as corresponding to theradiopaque tip of the guidewire, the features indicated by circles 46are classified as corresponding to the guiding catheter, the featuresindicated by squares 48 are classified as corresponding to the devicehead, and the features indicated by diamonds 50 are classified ascorresponding to the radiopaque markers that are typically disposedproximally to an IVUS device head. As described hereinabove, theclassification is typically performed using machine learning techniquessuch as SVM, Deep Believe Networks, Neural Networks, and/or RandomDecision Forest.

It is noted that typically, at this stage in the procedure, some of thefeatures classified as potentially being a given type of feature arefalse. Such false features are typically identified based upon themapping that is performed in phase 7 of the procedure, as described infurther detail hereinbelow.

Referring again to FIG. 1A, in phase 7 of the procedure, roadmap-mappingfunctionality 25 of processor 20 maps at least a portion of theidentified features of given images of the second set of extraluminalimages to locations along the roadmap pathway on the roadmap image. Forsome applications, an arrangement of two features within the image iscompared to a shape of at least a portion of the roadmap pathway.Typically, the mapping is performed by comparing an arrangement of threeor more of the features within the image to a shape of at least aportion of the roadmap pathway. For example, the roadmap-mappingfunctionality may determine vector(s) defined by pair(s) of features inthe image belonging to the second set of extraluminal images (e.g., avector defined by a radiopaque marker of the guidewire tip and aradiopaque marker of the endoluminal data-acquisition device head).Alternatively or additionally, the roadmap-mapping functionality maydetermine distance(s) between by pair(s) of features in the imagebelonging to the second set of extraluminal images (e.g., a vectordefined by a radiopaque marker of the guidewire tip and a radiopaquemarker of the endoluminal data-acquisition device head). Furtheralternatively or additionally, the roadmap-mapping functionality maydetermine an arrangement of vectors defined by two or more pairs offeatures in the image belonging to the second set of extraluminal images(e.g., by determining an angle between (a) a vector defined by aradiopaque marker of the guidewire tip and a radiopaque marker of theendoluminal data-acquisition device head, and (b) a vector defined bythe radiopaque marker of the endoluminal data-acquisition device headand the guiding catheter).

The arrangement of features within the image belonging to the second setof extraluminal images is compared to a shape of at least a portion ofthe roadmap pathway. For some applications, the arrangement of featureswithin the image belonging to the second set of extraluminal images iscompared to an arrangement of two or more locations within the roadmappathway. Typically, the arrangement of features within the imagebelonging to the second set of extraluminal images is compared to anarrangement of three or more locations within the roadmap pathway. Forexample, the roadmap-mapping functionality may compare the arrangementof features within the image belonging to the second set of extraluminalimages to vector(s) defined by pair(s) of points that are disposed onthe roadmap pathway. Or, the roadmap-mapping functionality may comparethe arrangement of features within the image belonging to the second setof extraluminal images to an arrangement of vectors defined by two ormore pairs of points that are disposed on the roadmap pathway.

Typically, between the acquisition of the roadmap image, and theacquisition of a given image belonging to the second set of extraluminalimages, the lumen has undergone changes in location and shape (e.g., dueto the subject's respiratory cycle, due to the subject's cardiac cycle,due to other movement of the subject, and/or due to the devices withinthe lumen having moved the lumen). Typically, by performing theabove-described comparison, the roadmap-mapping functionality determinesan estimated measure of a transformation (e.g., stretching, rotation,shrinking, etc.) that should be applied to the given extraluminal image,such that a best fit of the identified features within the extraluminalimage to the roadmap pathway is determined. Based upon the determinedtransformation, the roadmap-mapping functionality determines locationsof portions of the extraluminal image (e.g., features corresponding tothe endoluminal data-acquisition device) with respect to the roadmapimage, by applying the transformation to at least some points on theextraluminal image. In particular, the roadmap-mapping functionalitydetermines where, on the roadmap pathway within the roadmap image,respective features associated with the endoluminal device weredisposed, at the time when the extraluminal image was acquired.

For some applications, the mapping is performed using the followingtechnique:

Assuming:

q_(j) is the {x,y} coordinate of feature j in one of the second set ofextraluminal images (i.e., one of the extraluminal images that wasacquired during the data-acquisition by the endoluminal data-acquisitiondevice), where 1≦j≦m; and

-   -   p_(i) is the {x,y} coordinate of a general point along the        roadmap pathway within the roadmap image, where 1≦i≦n;

the mapping provides T: {1 . . . m}→{1 . . . n}.

Thus, the mapping maps feature q_(j) (in the extraluminal image) toposition P_(T(j)) (in the roadmap image).

As described hereinabove, typically, body lumens undergo variousdeformations, such as due to the cardiac cycle, respiration, and otherpossible movements of the subject. For some applications, in order toperform the mapping, the mapping functionality assumes that the generalshape of the lumen, and the relationships among features along thelumen, are generally preserved throughout the motion of the lumen. Inorder to find the desired index mapping, a deformation measure isdefined for each mapping T. The desired index mapping is obtained byminimizing the deformation measure.

Assuming that the vectors q_(j1)−q_(j2) and P_(T(j1))−P_(T(j2)) aresimilar, for all j₁, j₂, the deformation measure is defined by:

$\begin{matrix}{\sum\limits_{{1 \leq j_{1}},{j_{2} \leq m}}^{\;}\;{{C_{j_{1}j_{2}} \cdot {Similarity}}\mspace{11mu}( {{q_{j_{1}} - q_{j_{2}}},{p_{T{(j_{1})}} - p_{T{(j_{2})}}}} )}} & ( {{equation}\mspace{14mu} 1} )\end{matrix}$

where the coefficients C_(j) ₁ _(j) ₂ ≧0

For example, the similarity function may be defined in one of thefollowing ways:

$\begin{matrix}{{{Similarity}\mspace{11mu}( {u,v} )} = {{u - v}}^{2}} & ( {{equation}\mspace{14mu} 2} ) \\{{{{Similarity}\mspace{11mu}( {u,v} )} = {{\alpha\frac{( {{v} - {u}} )^{2}}{u}} + {\beta\frac{{{u - v}}^{2}}{u}}}}{\alpha,{\beta \geq 0}}} & ( {{equation}\mspace{14mu} 3} )\end{matrix}$

The deformation measure provided by equation 1 is computed using thesimilarity provided by equation 2 or equation 3, such as to providetransformation T. Thus, the location of each feature from theextraluminal image within the roadmap image is provided by P_(T(j)). Thetransformation that minimizes the deformation measure is typicallycomputed.

The effect of performing the above-described mapping algorithm is tocompare vectors defined by respective pairs of the identified featuresto vectors defined by respective pairs of locations within the roadmappathway. For example, the mapping algorithm may compare a vector definedby the spacing between the two stars at the bottom of FIG. 2D to avector defined by a pair of locations along the roadmap pathway. Forsome applications, the mapping algorithm compares a plurality of vectorsdefined by respective pairs of the identified features to a plurality ofvectors defined by respective pairs of locations within the roadmappathway. By performing the minimizing of the deformation measure, thealgorithm finds a way of best fitting the identified features within agiven extraluminal image to the roadmap pathway.

Reference is now made to FIG. 2E, which demonstrates, figuratively, theresult of performing the minimizing of the deformation measure. As shownin FIG. 2E, features that were identified in the extraluminal image havebeen mapped to locations on roadmap pathway 42 of the roadmap image insuch a manner as to minimize the deformation measure (i.e., in a mannerthat minimizes the extent to which the arrangement of identifiedfeatures must be deformed, in order that the identified features fitupon the roadmap pathway). It is noted that typically, the featuresidentified in the extraluminal image are not actually displayed upon theroadmap image, as shown in FIG. 2E. Rather, as stated above, FIG. 2Edemonstrates, figuratively, the result of performing the minimizing ofthe deformation measure. Based upon the above-described mappingprocedure, it is determined where, on the roadmap pathway in the roadmapimage, respective features associated with the endoluminal device weredisposed, at the time when the extraluminal image was acquired.

For some applications, in performing the mapping, one or more of thefollowing restrictions are applied such as to restrict the possiblelocations on the roadmap pathway to which a given feature of theextraluminal image may be mapped:

1) As described hereinabove, for some applications,landmark-classification functionality 22 classifies regions within theroadmap image as corresponding to locations within the roadmap imagewithin which features are likely to be. For some applications, inperforming the mapping, the roadmap-mapping functionality restricts themapping of features in an extraluminal image, such that a given featureis only allowed to be mapped to the corresponding region of the roadmapimage. Thus, for example, if a region of the roadmap image has beenclassified as corresponding to the guiding catheter, the roadmap-mappingfunctionality will not allow features that are identified as portions ofthe guidewire to be mapped to that region.

2) For given images of the second set of extraluminal images that areacquired in temporal proximity to one another, features of a given type(e.g., the endoluminal data-acquisition device head) are classified byassuming that, in each of the extraluminal images, the features of thegiven type must be in close proximity to each other. This is because itis assumed that since the extraluminal images were acquired in temporalproximity to one another, the features could not have moved by more thana given distance between the acquisitions of the respective images. Forsome applications, in determining the extent to which the features ofthe given type must be in close proximity to each other in theextraluminal images, the expected velocity of the endoluminaldata-acquisition device, and/or the expected foreshortening of theendoluminal data-acquisition device are accounted for.

3) For given images of the second set of extraluminal images that areacquired in temporal proximity to one another, the roadmap mappingfunctionality will only allow features of a given type (e.g., theendoluminal data-acquisition device head) within the respectiveextraluminal images to be mapped to locations that are in closeproximity to one another along the roadmap pathway. This is because itis assumed that since the extraluminal images were acquired in temporalproximity to one another, the features could not have moved along theroadmap pathway by more than a given distance between the acquisitionsof the respective images. For some applications, in determining theextent to which the features of the given type must be in closeproximity to each other along the roadmap pathway, the expected velocityof the endoluminal data-acquisition device, and/or the expectedforeshortening of the endoluminal data-acquisition device are accountedfor.

4) In performing the mapping, the roadmap-mapping functionality accountsfor known dimensions associated with the features. For example, by wayof illustration, the roadmap-mapping functionality may account for theknown separation between adjacent markers, the known length of theendoluminal data-acquisition device head, a known dimension of the guidecatheter, and/or a known dimension of the guidewire. The roadmap-mappingfunctionality restricts the mapping of features in an extraluminal imageto the roadmap pathway, by only allowing mapping that does not changethe known dimensions (and/or the relative dimensions) associated withthe features by more than a threshold amount.

5) Given features must be placed in a given order along the roadmappathway. For example, the guidewire distal tip must typically be thedistal-most feature, and the endoluminal data-acquisition device headmust be distal to the guide catheter, etc.

The result of performing the mapping on images belonging to the secondset of extraluminal images is typically that, for each of theextraluminal images to which the mapping is applied, an estimate isdetermined of where, at the time when the extraluminal image wasacquired, respective features associated with the endoluminal devicewere disposed upon the roadmap pathway. In particular, for each of theextraluminal images to which the mapping is applied, an estimate isdetermined of where, at the time when the extraluminal image wasacquired, the data-acquiring portion of the data-acquisition device(e.g., the endoluminal data-acquisition device head) was disposed uponthe roadmap pathway.

Typically, processor 20 determines which endoluminal data points wereacquired at the same time as respective extraluminal images. Forexample, a single computer (or two or more computers that aretime-synchronized) may operate both the extraluminal imaging and theendoluminal data-acquisition, and the computer may log the times atwhich extraluminal images and endoluminal data-points were acquired. Or,the processor may determine which endoluminal data points were acquiredat the same time as respective extraluminal images based upon knownframe rates at which the extraluminal images and the endoluminal datapoints are acquired. By determining an estimate of where, at the timewhen the extraluminal image was acquired, the data-acquiring portion ofthe data-acquisition device (e.g., the endoluminal data-acquisitiondevice head) was disposed upon the roadmap pathway, the processordetermines the location with respect to the roadmap pathway of theendoluminal data point that was acquired at the same time as theextraluminal image.

Referring again to FIG. 1A, in phase 8 of the procedure,location-interpolation functionality 26 of processor 20 performsinterpolation on the determined locations of the data-acquiring portionof the endoluminal data-acquisition device along the roadmap pathway,such as to determine the location of the data-acquiring portion of theendoluminal data-acquisition device along the roadmap pathway at anytime during the movement of the endoluminal data-acquisition device withrespect to the lumen (i.e., even at times between acquisitions ofextraluminal images belonging to the second set of extraluminal images).Typically, the interpolation is performed by optimizing a cost of atrajectory of the endoluminal data-acquisition device, throughout thepullback. The cost includes parameters such as maintaining a givenseparation between given features (e.g., between pairs of markers), thevelocity of the endoluminal data-acquisition device, movementcontinuity, and quality of the mapping of the guide catheter. Theresulting trajectory is smoothed and the result is a floating pointmodel index of endoluminal data-acquisition device locations along theroadmap pathway. For some applications, the location-interpolationfunctionality applies parameter estimation techniques to the determinedlocations of the data-acquiring portion of the endoluminaldata-acquisition device along the roadmap pathway. For example, temporalfiltration techniques, and/or outlier removal techniques may be appliedto the determined locations of the data-acquiring portion of theendoluminal data-acquisition device along the roadmap pathway.

For some applications, in order to perform the interpolation, theroadmap pathway is first calibrated using pathway-calibrationfunctionality 27 of processor 20. The pathway-calibration functionalitycalibrates the roadmap pathway by determining the relationship betweenthe physical dimension of a portion of the lumen and a number of pixelsin a portion of the roadmap pathway that corresponds to the portion ofthe lumen (e.g., the length in mm along the lumen, per pixel along theroadmap pathway). It is noted that typically, the calibration factorsassociated with respective portions of a lumen in an image varies, dueto respective portions of the lumen being disposed at respective angleswith respect to the extraluminal imaging device. Therefore, typically,the pathway calibration functionality determines a plurality of localcalibration factors along the roadmap pathway.

For some applications, the calibration is performed based upon knowndimensions associated with the features that are identified in theimages belonging to the second set of extraluminal images. For example,the pathway-calibration functionality may use a known separation betweenadjacent markers, the known length of the endoluminal data-acquisitiondevice head, a known dimension of the guide catheter, a known dimensionof a radiopaque marker, and/or a known dimension of the guidewire. Sincethe features are mapped to locations along the roadmap pathway (inaccordance with the techniques described hereinabove), thepathway-calibration functionality is able to determine at any givenlocation along the roadmap pathway a calibration factor associated withthat location by identifying the number of pixels within the portion ofthe roadmap pathway that correspond to the known dimension associatedwith the features.

For some applications, even if the actual dimensions associated withfeatures are not known, the pathway-calibration functionality determinesthe relative calibration factors of respective portions of the roadmappathway, based upon the relative number of pixels that a given featureor set of features occupy while the feature or set of features isdisposed within the respective portions of the pathway. For someapplications, the pathway-calibration functionality determines thecalibration factors of respective portions of the roadmap pathway basedupon a velocity at which one of the features is known to move. Forexample, if an endoluminal data-acquisition device is known to be pulledthrough the lumen (or pushed through the lumen) at a given speed, thepathway-calibration functionality may determine that, over a given timeinterval, the device moved through a given number of pixels along agiven portion of the roadmap pathway. In response thereto, theroadmap-calibration functionality determines the calibration factorassociated with the portion of the pathway. For some applications, ascale is placed along the roadmap pathway of the roadmap image basedupon the calibration.

For some applications, the calibration is performed using the followingtechnique, the goal of the calibration being to determine the distancebetween any two points along the roadmap pathway:

-   -   1. The roadmap pathway is denoted as a sequence of points        p_(i)=(x_(i), y_(i)), i=0, 1 . . . , n    -   2. The calibration is achieved if the distances from p₀ to p_(i)        are determined for i=0, 1 . . . , n.    -   3. The distances are denoted as d_(i) (d₀=0)    -   4. Using linear interpolation the pathway is parameterized by a        continuous parameter t ε[0, n] as follows:

x_(t) = (1 − α(t)) * x_(i(t)) + α(t) * x_(i(t) + 1)y_(t) = (1 − α(t)) * y_(i(t)) + α(t) * y_(i(t) + 1) where:$i = \{ {{\begin{matrix}\lbrack t\rbrack & {{{if}\mspace{14mu} 0} \leq t < n} \\{n - 1} & {{{if}\mspace{14mu} t} = n}\end{matrix}{\alpha(t)}} = {t - {i(t)}}} $

-   -   -   [t] is the integer part of t.

    -   5. For pairs of features in the images belonging to the second        set of extraluminal images that are disposed at a known physical        distance from each other, each mapping to the roadmap pathway        yields an equation. If the first feature of the pair is mapped        to (x_(t), y_(t)) and the second feature is mapped to (x_(s),        y_(s)) and the known distance between the pair of features is        denoted as r, the equation can be written as follows:        ((1−a(s))*d _(i(s)) +a(s)*d _(i(s)+1))−        ((1−a(t))*d _(i(t)) +a(t)*d _(i(t)+1))=r        -   (Without loss of generality, it is assumed that s>t.)        -   (If d₀ appears in the equation, it is replaced with 0.)

    -   6. The following regularization equations are generated:

${{W( {\frac{d_{i}}{{p_{i} - p_{i - 1}}} - \frac{d_{i + 1}}{{p_{i + 1} - p_{i}}}} )} = 0},{i = 1},{2\mspace{14mu}\ldots}\mspace{14mu},{n - 1}$

-   -   -   where W is the regularization weight.

    -   7. All the equations generated by steps (5) and (6) are solved        by a least square method.

    -   8. The solution yields d_(i), i=1, 2 . . . , n−1, from which the        distance between any two points on the roadmap pathway is        calculated, since the distance between two points p_(i) and        p_(j) can be computed from the estimated distances d_(i) and        d_(j) as |d_(i)−d_(j)|.

For some applications, based upon the interpolation of the locations ofthe endoluminal data-acquisition device along the roadmap pathway (and,optionally, calibration of the roadmap pathway), co-registrationfunctionality 28 of the processor co-registers respective endoluminaldata points to respective locations within the roadmap image.

Referring again to FIG. 1A, in phase 9 of the procedure,stack-generation functionality 29 of processor 20 generates a stack ofendoluminal data points, based upon the output of phase 8 of theprocedure. For some applications, the endoluminal data points includeendoluminal images, and the endoluminal images are arranged in an imagestack. Typically, the endoluminal image stack is generated by extractingan endoluminal image at regular intervals of length along the roadmappathway, and from each image, extracting a cross section of the image(typically, one line of pixels) and placing the cross section in thestack. Thus, the images are positioned at locations within the stackcorresponding to relative locations along the roadmap pathway within thelumen at which the images were acquired. For some applications, theendoluminal data points are functional endoluminal data points, and adisplay of the endoluminal data points is generated, in which theendoluminal data points are positioned at locations corresponding torelative locations within the lumen at which the endoluminal data pointswere acquired. Typically, the functional endoluminal data points aredisplayed in the stack by displaying a stack of indications of thefunctional endoluminal data points, locations of the indications withinthe stack corresponding to relative locations within the lumen at whichthe endoluminal data points were acquired. For example, numericalindications of the functional endoluminal data points may be displayedand/or representations of the functional endoluminal data points (whichmay be based upon a color-code, for example) may be displayed. For someapplications, indications of non-functional endoluminal data points aredisplayed in the described manner.

Reference is now made to FIG. 3A, which is a graph showing the locationalong a lumen (e.g., along the center line of the lumen) of adata-acquiring portion (e.g., the head) of an endoluminaldata-acquisition device, versus the frame numbers of the endoluminaldata points acquired by the data-acquisition device, during pullback ofthe data-acquisition device. Typically, even during automated pullbackof the data-acquisition device, the relative speed at which thedata-acquiring portion of the device moves with respect to the lumen,and, in some cases, the direction in which the data-acquiring portionmoves with respect to the lumen, varies over the course of the cardiaccycle, due to pulsation of the lumen. As shown on portion 52 of thegraph (which typically corresponds to a systolic phase of the cardiaccycle, or a portion thereof), in some cases, the data-acquiring portionof an endoluminal data-acquisition device moves forward (i.e., distally)with respect to the lumen during certain phases of the cardiac cycle,even during pullback (pullback generally being in a distal to proximaldirection).

Further typically, as a result of the data-acquiring portion movingforward with respect to the lumen, in some cases, two or moreendoluminal data points are acquired at a single location along thelumen. For example, as shown in FIG. 3A, frames x, y, and z are acquiredat a single location along the lumen. Frame x is acquired pre-systole,while the data-acquisition device is moving in a distal to proximaldirection with respect to the lumen, frame y is acquired during systole,while the data-acquisition device is moving in a proximal to distaldirection with respect to the lumen, and frame z is acquiredpost-systole, while the data-acquisition device is moving back past thesame location in a distal to proximal direction with respect to thelumen.

For some applications, manual pullback of the endoluminaldata-acquisition device is performed by an operator. In some cases,during manual pullback, the operator pushes the data-acquisition deviceforward at times in order to view a given region for a second time. As aresult, the data-acquisition device typically acquires a plurality ofendoluminal data points of given locations within the region. Forexample, a first data point may be acquired during the initial pullbackpast the location in the distal to proximal direction, a second datapoint may be acquired when the data-acquisition device is pushed forwardby the operator in the proximal to distal direction, and a third datapoint may be acquired when the data-acquisition device is, subsequently,pulled back past the location in the distal to proximal direction for asecond time.

Reference is now made to FIG. 3B, which is a graph showing the locationalong a lumen (e.g., along the center line of the lumen) of adata-acquiring portion of an endoluminal data-acquisition device, versusthe frame numbers of the endoluminal data points acquired by thedata-acquisition device, during pullback of the data-acquisition device.As indicated in FIG. 3B, for some applications, during some of thepullback of the endoluminal data-acquisition device (e.g., along portion54 of the graph), the data-acquisition device moves at a greater speedthan the regular pullback speed of the data-acquisition device, suchthat the location of the endoluminal data-acquisition device within theextraluminal images of the lumen cannot be determined by performingimage processing on the extraluminal images. For example, a region (suchas a narrow region) of the lumen may provide resistance to the pullbackof the data-acquisition device, such that the data-acquisition devicebecomes stuck for a period of time, following which the data-acquisitiondevice pulls back quickly from the region of resistance. If, by way ofexample, the extraluminal imaging device acquires an extraluminal imageonce every 1/15th of a second, and the data-acquisition device pullsback from an area of resistance at a speed of 150 mm/s, then this mayresult in there being no extraluminal image of the data-acquisitiondevice within a 10 mm section of the lumen. Thus, endoluminal datapoints that were acquired within the 10 mm section cannot be accuratelyco-registered to corresponding locations within the lumen in theextraluminal image.

Typically, stack-generation functionality 29 generates a corrected stackof endoluminal data points (e.g., endoluminal images) in which:

(a) there are one or more gaps in the stack at a portion of the stackcorresponding to a region within the lumen within which the endoluminaldata-acquisition device has not been imaged by the extraluminal imagingdevice;

(b) endoluminal data points that were acquired during forward motion ofthe endoluminal data-acquisition device are either rejected, or areappropriately placed within the stack; and/or

(c) at least one data point corresponding to a location along the lumenthat has two or more endoluminal data point corresponding thereto isrejected from being used in the stack.

Reference is now made to FIG. 3C, which shows an endoluminal image stack60 that has gaps 62 therein, in accordance with some applications of thepresent invention. Typically, as shown, the endoluminal image stack isshown on the same screen as the roadmap image 64. Further typically, anendoluminal image 66 corresponding to a location along the roadmappathway that is selected by the user is shown on the screen. For someapplications, processor 20 identifies regions of the lumen within whichthe endoluminal data-acquisition device has not been imaged by theextraluminal imaging device (e.g., due to the endoluminaldata-acquisition device moving through the region too quickly). Inresponse to a user selecting a location on the roadmap image that iswithin such a region, the processor generates an indication that thereis no endoluminal image corresponding to that location. For example, inresponse to the user selecting the location, the processor may notdisplay any endoluminal image, or the system may display an endoluminalimage corresponding to a location adjacent to the selected location, andgenerate an indication that this is the case. Alternatively, theprocessor may display an endoluminal image that was acquired by theendoluminal data-acquisition device while the data-acquisition devicemoved through the region, and generate an indication that the preciselocation within the region of the lumen corresponding to the endoluminalimage is not known.

It is noted that although FIG. 3C and the description thereof relate toendoluminal images, the scope of the present invention includes applyingsimilar techniques to other forms of endoluminal data points (e.g.,functional endoluminal data points), mutatis mutandis.

For some applications, the processor identifies regions of the lumenwithin which the endoluminal data-acquisition device has not been imagedby the extraluminal imaging device. In response thereto, the processordisplays gaps in the endoluminal image stack at the locations within thestack corresponding to the regions within the lumen, as shown in FIG.3C. For some applications, the processor assumes that the endoluminaldata-acquisition device moved through the region at a constant speed.The processor interpolates locations of the endoluminal images that wereacquired by the endoluminal data-acquisition device while thedata-acquisition device moved through the region, along the length ofthe portion within the endoluminal image stack that corresponds to theregion.

For some applications, processor 20 includes parameter-measurementfunctionality 30. In response to a user designating a portion of thestack of endoluminal data points, the parameter-measurementfunctionality determines a parameter of the portion of the roadmappathway corresponding to the designated portion of the stack, based uponthe co-registration of the stack to the roadmap pathway. For someapplications, the parameter-measurement functionality determines thelength of the portion of the roadmap pathway corresponding to thedesignated portion of the stack, based upon the co-registration of thestack to the roadmap pathway. For example, a user may designate aportion of an endoluminal data stack that contains a lesion, and inresponse thereto, the parameter-measurement functionality determines alength of the portion of the roadmap pathway corresponding to thedesignated portion of the stack, based upon the co-registration of thestack to the roadmap pathway. For some applications, theparameter-measurement functionality performs the aforementionedmeasurements, even if the endoluminal data stack that is displayed tothe user has not been corrected (to account for duplicate data pointsand gaps), as described hereinabove. Typically, length measurements thatare performed with respect to the roadmap pathway are more accurate thanif the length measurements were performed upon a raw data stack, interalia, because local calibration factors along the roadmap pathway areknown, as described hereinabove.

For some applications, length measurements are displayed on theendoluminal data stack. For some applications, measurements areautomatic. For some applications, measurements are performedinteractively by the user. For some applications, measurement of adifferent parameter (e.g., lumen diameter) is performed in a generallysimilar manner to that described above with respect to lengthmeasurement, mutatis mutandis. For some applications, a scale (or someother known dimension) presented on the endoluminal data stack providesa reference dimension for calibrating the measurements. For someapplications, based upon the co-registration of the endoluminal datastack to the roadmap image, a scale is displayed with reference to theendoluminal data stack.

For some applications, forward motion of the endoluminaldata-acquisition device that is (a) due to pulsation of the lumen,and/or (b) due to an operator of the data-acquisition device pushing thedata-acquisition device forward, is accounted for in order to facilitateco-registration of the endoluminal data points to an extraluminal image.Typically, in order to facilitate co-registration, the system identifiesredundant data points (i.e., data points that are not required becausethey are acquired at a location at which one or more additional datapoints are acquired), and rejects at least some of the redundant datapoints from being used for the co-registration, as described in furtherdetail hereinbelow.

For some applications, forward motion of the data-acquisition device isdetected by acquiring images of the endoluminal device within the lumen,and performing image processing on the images in order to determinelocations of the endoluminal device with respect to the lumen at thetime of the acquisition of respective endoluminal image frames, e.g., inaccordance with the techniques described hereinabove.

For some applications, forward motion of the endoluminal device isdetermined by performing the above-described mapping procedure.

For some applications, angiographic images of the data-acquisitiondevice within the lumen are acquired in the presence of contrast agent(which makes the lumen visible in the angiographic images), and theangiographic images are image processed in order to determine locationsof the endoluminal data-acquisition device marker with respect to thelumen at the time of the acquisition of respective endoluminal datapoints. Typically, image processing of angiographic images of thedata-acquisition device within the lumen is used to identify forwardmotion of the data-acquisition device that is (a) due to pulsation ofthe lumen, or (b) due to an operator of the data-acquisition devicepushing the data-acquisition device forward. This is because, in theangiographic images, the system typically identifies a visible movingportion of the endoluminal data-acquisition device (e.g., a radiopaquemarker on the data-acquiring portion). Using image processing, thesystem tracks the motion of the visible, moving portion of theendoluminal data-acquisition device with respect to the lumen. Thus,motion of the visible, moving portion of the data-acquisition devicewith respect to the lumen is identifiable in the angiographic images,irrespective of the cause of the motion.

For some applications, fluoroscopic images of the data-acquisitiondevice within the lumen are acquired in the absence of contrast agent,and the fluoroscopic images are image processed in order to determinelocations of the endoluminal data-acquisition device marker with respectto the lumen at the time of the acquisition of respective endoluminaldata points. For some applications, as described hereinabove, thelocation of a moving, visible portion of the endoluminaldata-acquisition device (e.g., a radiopaque marker on the data-acquiringportion of the endoluminal data-acquisition device) is determinedaccording to its distance along a guide wire along which thedata-acquisition device is inserted, the distance typically beingmeasured relative to the distal tip of a guiding catheter through whichthe guidewire and the data-acquisition device were previously inserted,and/or relative to radiopaque distal portion(s) (e.g., a radiopaquedistal tip) of the guidewire. For some applications, the endoluminaldata-acquisition device includes a portion that substantially does notmove with respect to the lumen during pullback, such as an insertionsheath. The location of a moving, visible portion of thedata-acquisition device is determined, via image processing, withreference to the portion of the device that substantially does not movewith respect to the lumen during pullback. For some applications, thelocation of a moving, visible portion of the endoluminaldata-acquisition device is determined with respect to a marker wire,over which the endoluminal data-acquisition device is inserted, inaccordance with the techniques described hereinabove.

Typically, image processing of fluoroscopic images of thedata-acquisition device within the lumen can be used to identify forwardmotion of the data-acquisition device that is due to an operator of thedata-acquisition device pushing the data-acquisition device forward.

For some applications, forward motion of the endoluminaldata-acquisition device that is caused by an operator pushing thedata-acquisition device forward is determined using a longitudinalposition/movement sensor coupled to apparatus through which theendoluminal data-acquisition device is inserted. Alternatively oradditionally, forward motion of the endoluminal data-acquisition devicethat is caused by an operator pushing the data-acquisition deviceforward is determined by performing the mapping procedure describedhereinabove with reference to FIG. 1A.

In response to determining that two or more endoluminal data pointscorrespond to the same location along the lumen due to forward motion ofthe data-acquisition device with respect to the lumen, at least one ofthe data points is not used for the co-display of the endoluminal datapoints with an extraluminal image of the lumen. For some applications,only the first endoluminal data point that was acquired at the locationis used for the co-display of the endoluminal data points with anextraluminal image of the lumen. For some applications, it is determinedwhich at least one of the two or more endoluminal data points thatcorrespond to the same location along the lumen was acquired duringforward motion of the data-acquisition device, and this data point isrejected from being used in the co-display. Alternatively oradditionally, another at least one of the two or more endoluminal datapoints that correspond to the same location along the lumen is rejectedfrom being used in the co-display.

For some applications, during pullback of the endoluminal imagingdevice, the subject's ECG signal is detected. Respective endoluminaldata points are identified as corresponding to the period in thesubject's cardiac cycle at the time when the data point was acquired,based upon the detected ECG signal (e.g., by indexing the image frameswith respect to the subject's ECG signal). For some applications, basedupon the identified correspondence, the system determines which of theendoluminal data points were acquired in a given period of the subject'scardiac cycle, such as at least a portion of systole, and these datapoints are not used for the co-display of the endoluminal data pointswith an extraluminal image of the lumen. For example, framescorresponding to at least a portion of the subject's ECG signal betweenthe S and T waves may be rejected from being used in the co-display.Typically, associating endoluminal data points with phases of thesubject's cardiac cycle (e.g., by indexing with respect to the subject'sECG signal) can be used to account for forward motion of the endoluminaldata-acquisition device that is caused by motion of the data-acquisitiondevice with respect to the lumen due to pulsation of the lumen that isdue to the subject's cardiac cycle.

For some applications, techniques described herein are used to accountfor the forward motion of the endoluminal data-acquisition device inorder to facilitate the generation of an endoluminal data stack, theforward motion of the data-acquisition device typically being (a) due topulsation of the lumen, and/or (b) due to an operator of thedata-acquisition device pushing the data-acquisition device forward.Typically, in order to facilitate generation of an endoluminal datastack, the system identifies redundant data points (i.e., data pointsthat are not required because they are acquired at a location at whichone or more additional data points are acquired), and rejects at leastsome of the redundant data points from being used in the endoluminaldata stack, as described in further detail hereinbelow. For someapplications, in response to determining that some of the data pointswere acquired during forward motion of the data-acquisition device, thesystem places the data points in order within the data stack, and/orre-orders data points in a data stack that has already been generated,such that the data points within the stack are placed in the correctorder. For some applications, the system indicates data points within adata stack that were acquired during forward motion of thedata-acquisition device, for example, by highlighting portions of thedata stack that were acquired during the forward motion.

For some applications, forward motion of the data-acquisition device isdetected by acquiring angiographic images or fluoroscopic images of thedata-acquisition device within the lumen, and performing imageprocessing on the angiographic images in order to determine locations ofthe endoluminal data-acquisition device marker with respect to the lumenat the time of the acquisition of respective endoluminal data points, asdescribed hereinabove. Typically, as described hereinabove, imageprocessing of angiographic images is used to identify forward motion ofthe data-acquisition device that is caused by (a) pulsation of thelumen, or (b) an operator of the data-acquisition device pushing thedata-acquisition device forward. Further typically, image processing offluoroscopic images is used to identify forward motion of thedata-acquisition device that is caused by an operator of thedata-acquisition device pushing the data-acquisition device forward. Forsome applications, forward motion of the endoluminal data-acquisitiondevice that is caused by an operator pushing the data-acquisition deviceforward is determined using a longitudinal position/movement sensorcoupled to apparatus through which the endoluminal data-acquisitiondevice is inserted. Alternatively or additionally, forward motion of theendoluminal data-acquisition device that is caused by an operatorpushing the data-acquisition device forward is determined by performingthe mapping procedure described hereinabove with reference to FIG. 1A.

For some applications, during pullback of the endoluminal imagingdevice, the subject's ECG signal is detected. Respective endoluminaldata points are identified as corresponding to the period in thesubject's cardiac cycle at the time when the data point was acquired,based upon the detected ECG signal (e.g., by indexing the data pointswith respect to the subject's ECG signal). For some applications, basedupon the identified correspondence, the system determines which of theendoluminal data points were acquired in a given period of the subject'scardiac cycle, such as at least a portion of systole. Typically,associating endoluminal data points with phases of the subject's cardiaccycle (e.g., by indexing with respect to the subject's ECG signal) canbe used to account for forward motion of the endoluminaldata-acquisition device that is caused by motion of the data-acquisitiondevice with respect to the lumen due to pulsation of the lumen that isdue to the subject's cardiac cycle.

For some applications, in order to generate the data stack, it isdetermined which data points were acquired during forward motion of theendoluminal data-acquisition device (e.g., based upon image processingof angiographic or fluoroscopic images of the device inside the lumen,or based upon associating the data points with respective phases of thesubject's cardiac cycle, such as, by indexing the data points withrespect to the subject's ECG signal), and, in response thereto, thosedata points are either rejected, or are appropriately placed within thestack. For some applications, in order to generate the stack it isdetermined which locations along the lumen have two or more endoluminaldata points corresponding thereto, and, in response thereto, at leastone of the data points corresponding to the location is rejected frombeing used in the endoluminal data stack. Typically, only the firstimaging frame to have been acquired at each location along the lumen isused in the data stack, and the other data points acquired at thelocation are rejected from being used in the data stack. Furthertypically, it is determined which at least one of the two or moreendoluminal data points that correspond to the same location along thelumen was acquired during forward motion of the data-acquisition device,and this data point is rejected from being used in the data stack.Alternatively or additionally, another at least one of the two or moreendoluminal data points that correspond to the same location along thelumen is rejected from being used in the data stack.

It is noted that some applications of the present invention have beendescribed with respect to an endoluminal data-acquisition device thatacquires data points while moving generally in a distal to proximaldirection (i.e., during pullback of the data-acquisition device), butthat experiences some movement in a proximal to distal direction. Thescope of the present invention includes applying the techniquesdescribed herein to an endoluminal data-acquisition device that acquiresdata points while moving generally in a proximal to distal direction(i.e., while the data-acquisition device is being pushed forward throughthe lumen), but that experiences some movement in a distal to proximaldirection, mutatis mutandis.

For some applications, in order to perform the above-describedtechniques, processor 20 includes (a)duplicate-data-point-identification functionality 31 configured todetermine that, at at least one location, two or more endoluminal datapoints were acquired by the endoluminal data-acquisition device, (b)data-point-selection functionality 32 configured to generate an outputusing a portion of the plurality of endoluminal data points of the lumenacquired using the endoluminal data-acquisition device, by using only asingle data point corresponding to the location, and (c) display-drivingfunctionality 33 configured to drive a display to display the output.

For some applications, the processor includes (a)direction-determination functionality 34 configured to determine that,while acquiring at least one of the endoluminal data points, theendoluminal data-acquisition device was moving in a second directionthat is opposite to the first direction, (b) output-generationfunctionality 35 configured, in response to the determining, to generatean output using at least some of the plurality of endoluminal datapoints of the lumen acquired using the endoluminal data-acquisitiondevice, and (c) display-driving functionality 33 configured to drive adisplay to display the output.

For some applications, typically in order to facilitate co-registrationof endoluminal data points to one or more extraluminal images, during(manual or automatic) pullback of an endoluminal data-acquisitiondevice, extraluminal images of the data-acquisition device within thelumen are acquired. Image processing is performed on the extraluminalimages in order to determine locations of the endoluminaldata-acquisition device marker with respect to the lumen at the time ofthe acquisition of respective endoluminal data points, e.g., inaccordance with the techniques described hereinabove. As describedhereinabove, for some applications, angiographic images of thedata-acquisition device within the lumen are acquired in the presence ofcontrast agent (which makes the lumen visible in the angiographicimages), and the angiographic images are image processed in order todetermine locations of the endoluminal data-acquisition device markerwith respect to the lumen at the time of the acquisition of respectiveendoluminal data points. Alternatively or additionally, fluoroscopicimages of the data-acquisition device within the lumen are acquired inthe absence of contrast agent, and the fluoroscopic images are imageprocessed in order to determine locations of the endoluminaldata-acquisition device marker with respect to the lumen at the time ofthe acquisition of respective endoluminal data points.

For some applications, as described hereinabove, the location of amoving, visible portion of the endoluminal data-acquisition device(e.g., a radiopaque marker on the data-acquiring portion of theendoluminal data-acquisition device) is determined according to itsdistance along a guide wire along which the data-acquisition device isinserted, the distance typically being measured relative to the distaltip of a guiding catheter through which the guidewire and thedata-acquisition device were previously inserted, and/or relative toradiopaque distal portion(s) (e.g., a radiopaque distal tip) of theguide wire. For some applications, the endoluminal data-acquisitiondevice includes a portion that substantially does not move with respectto the lumen during pullback, such as an insertion sheath. The locationof a moving, visible portion of the data-acquisition device isdetermined, via image processing, with reference to the portion of thedevice that substantially does not move with respect to the lumen duringpullback. For some applications, the location of a moving, visibleportion of the data-acquisition device is determined with respect to amarker wire, over which the data-acquisition device is inserted, inaccordance with the techniques described hereinabove.

For some applications, motion of the data-acquisition device withrespect to the lumen is determined by performing the above-describedmapping procedure.

For some applications, during some of the pullback of the endoluminaldata-acquisition device, the data-acquisition device moves at adifferent speed than the regular pullback speed of the data-acquisitiondevice. For some applications, during some of the pullback of theendoluminal data-acquisition device, the data-acquisition device movesat a greater speed than the regular pullback speed of thedata-acquisition device, such that the location of the endoluminaldata-acquisition device within the extraluminal images of the lumencannot be determined by performing image processing on the extraluminalimages. For example, a region (such as a narrow region) of the lumen mayprovide resistance to the pullback of the data-acquisition device, suchthat the data-acquisition device becomes stuck for a period of time,following which the data-acquisition device pulls back quickly from theregion of resistance. If, by way of example, the extraluminal imagingdevice acquires an extraluminal image once every 1/15th of a second, andthe data-acquisition device pulls back from an area of resistance at aspeed of 150 mm/s, then this may result in there being no extraluminalimage of the data-acquisition device within a 10 mm section of thelumen. Thus, endoluminal data points that were acquired within the 10 mmsection cannot be accurately co-registered to corresponding locationswithin the lumen in the extraluminal image. For some applications, thesystem identifies regions of the lumen within which the endoluminaldata-acquisition device has not been imaged by the extraluminal imagingdevice (e.g., due to the endoluminal data-acquisition device movingthrough the region too quickly). In response to a user selecting alocation on an extraluminal image that is within such a region, thesystem generates an indication that there is no endoluminal data pointcorresponding to that location. For example, in response to the userselecting the location, the system may not display any endoluminal datapoint, or the system may display an endoluminal data point correspondingto a location adjacent to the selected location, and generate anindication that this is the case. Alternatively, the system may displayan endoluminal data point that was acquired by the endoluminaldata-acquisition device while the data-acquisition device moved throughthe region, and generate an indication that the precise location withinthe region of the lumen corresponding to the endoluminal data point isnot known.

For some applications, processor 20 identifies regions of the lumenwithin which the endoluminal data-acquisition device has not been imagedby the extraluminal imaging device. In response thereto, the processordisplays gaps in the endoluminal data stack (e.g., the endoluminal imagestack) at the locations within the stack corresponding to the regions ofthe lumen. Such an endoluminal image stack is shown in FIG. 3C. For someapplications, the system assumes that the endoluminal data-acquisitiondevice moved through the region at a constant speed. The systeminterpolates locations of endoluminal data points that were acquired bythe endoluminal data-acquisition device while the data-acquisitiondevice moved through the region, along the length of the portion of theendoluminal data stack that corresponds to the region.

For some applications, techniques described herein (e.g., techniquesdescribed with reference to FIG. 3C) are performed by a system thatincludes at least one processor, for use with an endoluminaldata-acquisition device that acquires a plurality of endoluminal datapoints of a lumen of a body of a subject while being moved through thelumen generally in a first direction with respect to the lumen, and anextraluminal imaging device that images the endoluminal data-acquisitiondevice while the endoluminal data-acquisition device is moved throughthe lumen. For some applications, the processor includes (a)region-identification functionality 36 configured to identify regions ofthe lumen within which the endoluminal data-acquisition device has notbeen imaged by the extraluminal imaging device, (b) output-generationfunctionality 35 configured to generate an output that indicates thatthe endoluminal data-acquisition device has not been imaged by theextraluminal imaging device within the region, and (c) display-drivingfunctionality 33 configured to drive a display to display the output.

For some applications, the processor includes (a) region-identificationfunctionality 36 configured to identify regions of the lumen withinwhich the endoluminal data-acquisition device has not been imaged by theextraluminal imaging device, (b) stack-generation functionality 29configured to generate endoluminal data stack using the plurality ofendoluminal data points, and, in response to the identifying, togenerate a gap in the endoluminal data stack at a portion of the stackcorresponding to the region within the lumen, and (c) display-drivingfunctionality 33 configured to drive a display to display theendoluminal data stack.

Typically, as described hereinabove, stack-generation functionality 29is configured to generate an endoluminal data stack in which:

(a) there is at least one gap in the endoluminal data stack at a portionof the stack corresponding to a region of the lumen within which theendoluminal data-acquisition device was not imaged by the extraluminalimaging device;

(b) endoluminal data points that were acquired during forward motion ofthe endoluminal data-acquisition device (e.g., as determined based uponimage processing of angiographic or fluoroscopic images of the deviceinside the lumen, or based upon associating the frames with respectivephases of the subject's cardiac cycle, such as, by indexing the frameswith respect to the subject's ECG signal) are either rejected, or areappropriately placed within the stack; and/or

(c) at least one data point corresponding to a location along the lumenthat has two or more endoluminal data points corresponding thereto isrejected from being used in the endoluminal data stack.

Typically, while an endoluminal data-acquisition device is moved througha lumen (e.g., while an IVUS probe is pulled back or pushed forwardthrough a blood vessel), the device undergoes non-longitudinal motion.For example, the data-acquiring portion of the device (e.g., the head ofthe device) typically moves in an axial direction, rotates about thelongitudinal axis of the device, and/or becomes tilted. For someapplications, stack-generation functionality 29 determines thatendoluminal data points are not aligned with each other due tonon-longitudinal motion undergone by a portion of the endoluminaldata-acquisition device with respect to the lumen, between acquisitionsof respective endoluminal data points. In response thereto,stack-generation functionality 29 aligns the endoluminal data pointswith each other, to account for the non-longitudinal motion undergone bythe portion of the endoluminal data-acquisition device.

For some applications, techniques are used for generating a stack ofendoluminal data points (e.g., endoluminal images) in which non-uniformlongitudinal motion of a portion of the endoluminal data-acquisitiondevice is accounted for, as described in US 2012/0230565, WO 10/058,398,US 2012/0004537 and/or WO 12/014,212, all of which applications areincorporated herein by reference.

For some applications, in order to determine the angular orientation ofthe portion of the data-acquisition device with respect to the lumen atthe time of the acquisition of respective endoluminal data points, anasymmetrically-shaped radiopaque marker that is visible in extraluminalimages (e.g., angiographic or fluoroscopic images) of the lumen isdisposed on the data-acquiring portion (e.g., the imaging head) of theendoluminal data-acquisition device. Alternatively or additionally, themarker may be disposed asymmetrically with respect to the longitudinalaxis of the data-acquiring portion of the endoluminal data-acquisitiondevice. During the acquisition of endoluminal data points by theendoluminal data-acquisition device, extraluminal images are acquired ofthe endoluminal data-acquisition device within the lumen. Imageprocessing is applied to the fluoroscopic images in order to determinethe angular orientation of the data-acquiring portion of thedata-acquisition device with respect to the lumen at the time of theacquisition of respective endoluminal data points, typicallyautomatically and typically on-line, in accordance with techniquesdescribed herein.

For some applications, endoluminal data points (e.g., images) arealigned with each other in the stack, using image processing techniques.For example, stack-generation functionality 29 may identify a region ofone of the endoluminal images as having a given characteristic (e.g.,being lighter than the surrounding regions). Stack-generationfunctionality 29 may then search for a region in an adjacent endoluminalimage that has the same characteristic, and may align the adjacent imageframes by aligning the regions of each of the image frames.

For some applications, endoluminal data points that are indicative offunctional characteristics of the lumen are aligned with each other toaccount for non-longitudinal motion undergone by a portion of theendoluminal data-acquisition device with respect to the lumen, betweenacquisitions of respective endoluminal data points. For someapplications, a sensor is coupled to the data-acquiring portion of theendoluminal data-acquisition device, and the sensor is used to determinethe non-longitudinal orientation of the data-acquiring portion at timesat which respective endoluminal data points are acquired.

For some applications, the aforementioned techniques are applied inorder to account for unintentional rotation (typically, roll) of aportion of the endoluminal data-acquisition device with respect to thelumen, due to pulsation of the lumen, for example. For someapplications, the aforementioned techniques are applied in order tofacilitate the generation of an endoluminal image stack, in which theimages that comprise the stack are correctly rotationally aligned.Alternatively or additionally, the aforementioned techniques are appliedto determine the orientation with respect to each other of vessels thatappear in the endoluminal images.

Referring again to FIG. 1A, in phase 10, the endoluminaldata-acquisition device is typically retrieved from the designated site(and, further typically, withdrawn from the lumen), in order toaccommodate the insertion of an endoluminal device (e.g., an endoluminaltherapeutic device) into the lumen.

In phase 11, while observing an extraluminal image (and typically theroadmap image) of the luminal segment comprising the designatedlocation, one or more locations along that segment are indicated by auser input device. In response thereto, the previously-acquiredendoluminal data points (e.g., images) corresponding to the one or morelocations are displayed. For some applications, the user input device isused to select the one or more locations. Typically, the user designatesa location using the user input device, and, in response thereto,typically automatically and on-line, the system identifies a locationalong the lumen (typically along the roadmap pathway) as correspondingto the designated location, and retrieves and displays a correspondingendoluminal data point (e.g., image).

Alternatively or additionally, by observing an angiogram frame side byside with endoluminal image frames of the luminal segment comprising thedesignated location, one or more locations along the segment areindicated by a user input device with respect to endoluminal imagingdata.

For some applications, the user indication is made upon the endoluminalimage stack. For some applications, the processor generates a virtualdevice inside the endoluminal image stack in response to a user input.For example, a user may wish to generate an image of a device (e.g., aballoon, a stent, or a valve) inside an endoluminal image stack that hasbeen generated in accordance with the techniques described hereinabove.The image stack has typically been (a) corrected to show gaps in thestack, (b) corrected to remove duplicate endoluminal images, (c)corrected to account for the non-longitudinal motion undergone by theendoluminal data-acquisition device, and/or (d) calibrated with respectto physical dimensions of the lumen, in accordance with the techniquesdescribed hereinabove. Thus, the endoluminal image stack typicallyprovides to the user a representation of a cross section of the lumenthat is calibrated with respect to physical dimensions of the lumen. Forsome applications, the user places a virtual device within theendoluminal image stack, and modifies dimensions of the device in orderto determine suitable dimensions for a physical device that is to beplaced inside the lumen.

For some such applications, a baseline extraluminal image (typically theroadmap image) is selected for lesion analysis and a lesion is selectedby the physician for therapy. The physician then generates an indicationof a desired location for placement of the endoluminal therapeutic toolon the baseline image, e.g., by virtually placing an endoluminaltherapeutic tool (e.g., a balloon, a stent, or a valve) in the baselineimage, by marking a target line in the baseline image, and/or by markingdistal and proximal marking lines in the baseline image.

For some applications, the user indication is made by browsing throughthe endoluminal images. In response to receiving the user indication,the location along the lumen (e.g., along the luminal center line)within the angiogram corresponding to the location indicated withrespect to an endoluminal image or the endoluminal image stack isdetermined and indicated.

Typically, a clinical diagnosis is facilitated by a user viewingpreviously-acquired endoluminal images corresponding to the one or morelocations selected on extraluminal images of the luminal segment, or bythe user viewing indications of locations on an extraluminal image thatcorrespond to one or more locations selected with respect to endoluminalimages or an endoluminal image stack, as described with reference tophase 11. Alternatively, a clinical diagnosis is made by the userreviewing the extraluminal images and/or the endoluminal data (and/or byreviewing other data), without performing phase 11. Typically, atherapeutic process, such as the one described in phase 12 and beyond,is performed based upon the clinical diagnosis made by the user.

In phase 12, a second endoluminal device (e.g., a diagnostic device, asecond endoluminal data-acquisition device, or a therapeutic endoluminaldevice) is moved toward the designated location under real-timeextraluminal imaging. Typically, stabilization (and optionally alsoenhancement) is applied, typically on-line and typically automatically,to the extraluminal image stream.

In phase 13, using the above-described mapping algorithm, the currentlocation of the second endoluminal device, determined viaimage-processing that is performed on the current extraluminal images,is mapped to the roadmap image. The current device location is indicatedin the roadmap image. Typically, in cases in which the second device isa therapeutic device, in response to the mapping, the physician deploysthe endoluminal therapeutic device, in response to the roadmap imageindicating that the mapped location of the therapeutic device within theroadmap image corresponds to the desired location of the device asindicated within the roadmap image.

It is noted that, in general, the scope of the present inventionincludes using the technique of mapping extraluminal images of a deviceinside a lumen to a baseline roadmap image of the lumen (using thetechniques described hereinabove with respect to phases 6-7), in orderto determine the location of a device with respect to the roadmap imageat times corresponding to respective extraluminal images, and generatingan output in response thereto. Although with reference to FIG. 1A, themapping is described as being performed for the purpose ofco-registering endoluminal images to an extraluminal roadmap image, forsome applications, in response to the mapping, the on-line location ofan endoluminal device with respect to the roadmap pathway is determinedand is displayed upon the roadmap. For example, the on-line location ofthe endoluminal data-acquisition device with respect to the roadmappathway may be determined, by mapping an on-line extraluminal image ofthe endoluminal data-acquisition device inside the lumen to the roadmapimage using the mapping technique described hereinabove. In responsethereto, the on-line location of the endoluminal data-acquisition devicemay be displayed upon the roadmap image. Alternatively or additionally,the on-line location of a different endoluminal device (e.g., theendoluminal therapeutic device) with respect to the roadmap pathway maybe determined, by mapping an on-line extraluminal image of theendoluminal device inside the lumen to the roadmap image using themapping technique described hereinabove. In response thereto, theon-line location of the endoluminal device may be displayed upon theroadmap image. Further alternatively or additionally, the location ofthe endoluminal data-acquisition device with respect to the roadmappathway may be determined, and in response thereto, endoluminal datapoints may be co-registered to locations along the roadmap pathway, inaccordance with the techniques described hereinabove.

It is noted that, in general, the scope of the present inventionincludes determining a plurality of local calibration factors associatedwith respective locations on a roadmap image (using the techniquesdescribed hereinabove with respect to phase 8), and generating an outputin response thereto. Typically, the local calibration factors aredetermined based upon known dimensions associated with features that areidentified in images belonging to a second set of extraluminal images,in accordance with the techniques described hereinabove. Although withreference to FIG. 1A, the mapping is described as being performed forthe purpose of co-registering endoluminal images to an extraluminalroadmap image, for some applications, based upon the local calibrationfactors, the on-line location of an endoluminal device with respect tothe roadmap pathway is determined and is displayed upon the roadmap. Forexample, the on-line location of the endoluminal data-acquisition devicewith respect to the roadmap pathway may be determined, by mapping anon-line extraluminal image of the endoluminal data-acquisition deviceinside the lumen to the roadmap image, using the mapping techniquedescribed hereinabove, and based upon determined local calibrationfactors associated with the roadmap image. In response thereto, theon-line location of the endoluminal data-acquisition device may bedisplayed upon the roadmap image. Alternatively or additionally, theon-line location of a different endoluminal device (e.g., theendoluminal therapeutic device) with respect to the roadmap pathway maybe determined, by mapping an on-line extraluminal image of theendoluminal device inside the lumen to the roadmap image, using themapping technique described hereinabove, and based upon determined localcalibration factors associated with the roadmap image. In responsethereto, the on-line location of the endoluminal device may be displayedupon the roadmap image. Further alternatively or additionally, thelocation of the endoluminal data-acquisition device with respect to theroadmap pathway may be determined, using the mapping technique describedhereinabove, and based upon determined local calibration factorsassociated with the roadmap image. In response thereto, endoluminal datapoints may be co-registered to locations along the roadmap pathway, inaccordance with the techniques described hereinabove.

Data points (e.g., images) that were previously acquired by theendoluminal data-acquisition device at or near the location areretrieved and associated, typically on-line and typically automatically,with the extraluminal imaging, while the device is at or near the samelocation.

In phase 14, data points (e.g., images) that were previously acquired bythe endoluminal data-acquisition device at or near the location aredisplayed together with the extraluminal imaging. Typically, data pointsare displayed that correspond to the current location of the endoluminaltherapeutic device (as determined in phase 9). Typically, phases 13 and14 are performed in real-time with respect to phases 11 and 12. Thus,while the endoluminal therapeutic device is at respective currentlocations inside the lumen, the location of the device is determined,and the endoluminal data points associated with the location areretrieved and displayed.

For some applications, data acquired by a first endoluminal modality(e.g., IVUS) are co-registered with the roadmap image, in accordancewith the techniques described hereinabove. Subsequently, data acquiredby a second endoluminal modality (e.g., OCT) are co-registered with theroadmap image, in accordance with the applications describedhereinabove. Consequently, due to both data sets being co-registeredwith the roadmap image, the two data sets are co-registered to oneanother. For some applications, the two endoluminal data sets aredisplayed as overlaid or otherwise merged with one another.

For some applications, in response to determining the current locationof the second endoluminal device with respect to the roadmap pathway,the display-driving functionality is configured to drive a display todisplay an image of the second endoluminal device at the correspondinglocation within the endoluminal image stack. In accordance withrespective applications, a virtual image of the second device, or a realimage of the second device, is displayed within the endoluminal imagestack.

Reference is now made to FIG. 4, which is schematic illustration of ascreen on which an IVUS image 83 is displayed, in accordance with someapplications of the present invention. Typically, upon receiving anindication from the user of a location along the lumen (e.g., along theluminal center line 82, for example, by the user pointing cursor 81 to alocation on the screen, and the system determining a location along thecenter line corresponding to the location), IVUS image 83 which waspreviously acquired at that location is displayed. For someapplications, an IVUS stack comprising data from IVUS images that werepreviously acquired along a section of the lumen (e.g., along a sectionof center line 82) of which the user-indicated location is a middlepoint or one of the end points, is displayed. For some applications, anIVUS stack comprising data from IVUS images that were previouslyacquired between two user-indicated locations along the lumen (e.g.,along center line 82) is displayed. For some applications, similartechniques are performed using an endoluminal imaging modality otherthan IVUS.

For some applications, a three-dimensional “tunnel-like” reconstructionof the IVUS images of the vessel (or a section thereof, such as thosecorresponding to the longitudinal section between the current locationsof the proximal and distal markers of the balloon/stent) is generatedand displayed. For some applications, the IVUS images are overlaid onthe fluoroscopic images. For some applications, the IVUS images arefused with the fluoroscopic images. For some applications, a combinationof the aforementioned display techniques is applied. For someapplications, an indication of the motion range of the balloon/stentrelative to the lumen, resulting from the cardiac cycle, is displayed inconjunction with any of the aforementioned displays of the IVUS images.For some applications, such an indication is generated and/or displayedin accordance with embodiments of US 2010/0222671 to Cohen, which isincorporated herein by reference. For some applications, similartechniques are performed using an endoluminal imaging modality otherthan IVUS.

It is noted that in applying any of the techniques described hereinabovefor associating endoluminal images with respective locations along thelumen, the system typically accounts for a known offset between thelocation of the moving, visible portion of the endoluminaldata-acquisition devices (e.g., a radiopaque marker), and the locationof the data-acquiring portion of the probe (e.g., the ultrasoundtransducer, in the case of an IVUS probe).

It is noted that some of the techniques described hereinabove forassociating endoluminal images with respective locations along the lumenare described with reference to an endoluminal data-acquisition devicethat acquires endoluminal data points during pullback of the device. Thescope of the present invention includes applying any of the techniquesdescribed hereinabove for associating endoluminal data points withrespective locations along the lumen to an endoluminal data-acquisitiondevice that acquires endoluminal data points during insertion andadvancement of the device through the lumen (e.g., when images areacquired from an endobronchial airway), mutatis mutandis.

For some applications, pullback of the endoluminal data-acquisitiondevice is performed in the course of a continuous injection of contrastagent performed under fluoroscopic imaging. For example, the endoluminaldata-acquisition device may be an OCT probe, the image acquisition ofwhich typically requires concurrent flushing of the lumen, in order toremove blood from the lumen, the blood interfering with the OCT imaging.Furthermore, contrast agent highlights the lumen and facilitatesangiographic imaging of the lumen. Still furthermore, for someapplications, the presence of contrast agent in the lumen facilitatesacquisition of OCT data. Therefore, typically, during endoluminalimaging with an OCT probe, contrast agent is continuously injected intothe lumen. In addition, the pullback of the OCT probe is typicallyperformed rapidly relative to the pullback of an IVUS probe, and theframe acquisition rate of the OCT probe is typically greater than thatof an IVUS probe.

For some applications, a procedure is performed in order to co-registerOCT images to an extraluminal image of the lumen, the procedureincluding at least some of the following steps:

1) The OCT probe is inserted under extraluminal fluoroscopic imaging.The OCT probe typically includes one or more radiopaque portions thatmove in conjunction with the data-acquiring portion (e.g., the head) ofthe probe, and that have a known dimension associated therewith. Forexample, the data-acquiring portion of the probe itself is typicallyradiopaque and has a known dimension. In addition, the probe may haveradiopaque markers that move in conjunction with the data-acquiringportion of the probe and that are separated from each other by a knowndistance. Typically, the one or more radiopaque portions are identifiedin the extraluminal fluoroscopic images.

2) Pullback of the OCT probe commences at a known and steady speed(typically by means of automated pullback), in conjunction with contrastagent injection performed under angiographic imaging. The image slicesgenerated by the OCT along the pullback are recorded and stored togetherwith an indication of the time of acquisition and/or the frame number ofeach of the images.

3) A roadmap image is selected from the angiographic sequence, inaccordance with the techniques described hereinabove. A roadmap pathwayis designated within the roadmap image, in accordance with thetechniques described hereinabove.

4) The fluoroscopic images that were acquired during the insertion ofthe OCT probe are mapped to the roadmap image, in accordance with thetechniques described hereinabove. Local calibration factors along theroadmap pathway are determined, based upon the mapping, in accordancewith the techniques described hereinabove.

5) The starting location along the roadmap pathway at which the probewas disposed at the initiation of the pullback of the probe isdetermined.

6) The pullback speed of the OCT probe is known. In addition, the framerate of the OCT probe is known. Therefore, the distance along the lumenbetween adjacent OCT images is known. Furthermore, the local calibrationfactors for calibrating pixels along the roadmap pathway to the physicaldimensions of the lumen are typically known (based upon implementing theabove-described techniques). Thus, for any one of the OCT frames, thedistance from the starting location at which the OCT frame was acquiredis determined, based upon the speed at which the endoluminaldata-acquisition device was moved through the lumen, the frame rate atwhich the endoluminal data points were acquired, and the localcalibration factors associated with the respective locations within thelumen. For example, if it is known, based upon the speed of the pullbackand the frame rate, that images are acquired at intervals of 0.25 mm,then it is determined that the OCT image corresponding to a locationthat is 15 mm along the lumen from the pullback starting location is the60th image frame. Thus, for some applications, co-registrationfunctionality 28 of processor 20 co-registers respective endoluminaldata points to respective locations within the roadmap image, by (a)identifying the starting location of the endoluminal data-acquisitiondevice in the roadmap image, and (b) determining a distance from thestarting location at which respective endoluminal data points wereacquired, based upon the speed at which the endoluminal data-acquisitiondevice was moved through the lumen, the frame rate at which theendoluminal data points were acquired, and the local calibration factorassociated with the respective portions of the roadmap image.

7) Based upon the co-registering of the OCT images to the roadmap image,techniques as described hereinabove for displaying endoluminal images inconjunction with extraluminal images are performed. For example, inresponse to a user indicating a location along the lumen on anextraluminal image, the corresponding OCT image may be displayed. Or, anOCT image stack may be corrected using the techniques describedhereinabove, and may then be displayed, and/or used to facilitate lengthmeasurements along the roadmap pathway. For some applications, lengthmeasurements are displayed on the OCT image stack. For someapplications, measurements are automatic. For some applications,measurements are performed interactively by the user. For someapplications, a scale (or some other known dimension) presented on theOCT images provides a reference dimension for calibrating themeasurements. For some applications, a virtual device (e.g., a stent, aballoon, and/or a valve) is displayed a upon the OCT image stack,typically at a user-indicated location.

It is noted that although the above-described technique was describedwith respect to OCT imaging, the scope of the present invention includesperforming the above-described technique using other endoluminal imagingmodalities (such as IVUS and/or other imaging techniques describedhereinabove), mutatis mutandis. It is further noted that although theabove-described technique was described with respect to endoluminalimages that are acquired during pullback of the device, the scope of thepresent invention includes performing the above-described techniqueusing endoluminal images that are acquired while the imaging device isadvanced through the lumen, mutatis mutandis. It is still further notedthat although in the above-described technique, step (1) is described asbeing performed before steps (2) and (3), the scope of the presentinvention includes performing steps (2) and (3) and, subsequently,performing step (1).

For some applications, data acquired by a first endoluminal modality(e.g., IVUS) are co-registered with the fluoroscopic image stream, inaccordance with the applications described hereinabove. Subsequently,data acquired by a second endoluminal modality (e.g., OCT) areco-registered with the fluoroscopic image stream, in accordance with theapplications described hereinabove. Consequently, due to both data setsbeing co-registered with the fluoroscopic image stream, the two datasets are co-registered to one another. For some applications, the twoendoluminal data sets are displayed overlaid or otherwise merged withone another.

For some applications, generally similar steps to those described withreference to FIG. 1A are performed, except for the followingdifferences. In phase 12, instead of a therapeutic endoluminal device(e.g., a treatment catheter) being inserted into the lumen, a secondendoluminal data-acquisition device is inserted into the lumen.Typically, the first and second endoluminal data-acquisition devicesacquire endoluminal images using respective imaging modalities. Forexample, in phase 1, an IVUS probe may be inserted into the lumen, andin phase 12 an OCT probe may be inserted into the lumen, or vice versa.

The current location of the second endoluminal data-acquisition deviceis determined, for example, using any of the techniques described herein(such as, by performing image processing on extraluminal images of thesecond endoluminal data-acquisition device inside the lumen).Endoluminal images which were previously acquired using the firstdata-acquisition device at the current location of the secondendoluminal data-acquisition device are retrieved and displayed,typically on-line and typically automatically.

Typically, the endoluminal images which were acquired using the firstdata-acquisition device at the current location of the secondendoluminal data-acquisition device are displayed together withendoluminal images that are being acquired in real-time by the secondendoluminal data-acquisition device, while the second endoluminaldata-acquisition device is at the current location. For someapplications, endoluminal images that are acquired in real-time by thesecond endoluminal data-acquisition device, while the second endoluminaldata-acquisition device is at the current location, are displayedtogether with an indication of the current location of the secondendoluminal data-acquisition device with respect to an endoluminal imagestack generated using endoluminal images that were previously acquiredby the first endoluminal data-acquisition device. For some applications,using the above-described technique, data acquired by first and secondendoluminal data-acquisition devices are registered with respect to oneanother, and the co-registered data are displayed subsequent totermination of the acquisition of endoluminal images by both the firstand the second endoluminal data-acquisition devices. For someapplications, endoluminal images corresponding to the current locationof the second endoluminal data-acquisition device that were acquired bythe first endoluminal data-acquisition device and/or by the secondendoluminal data-acquisition device are co-displayed with an indicationof the current location of the second endoluminal data-acquisitiondevice on an extraluminal image of the lumen, using the techniquesdescribed herein.

For some applications, locations along the lumen of an endoluminaldata-acquisition device associated with a first endoluminaldata-acquisition modality (e.g., IVUS) are identified as correspondingto respective endoluminal data points of the first data-acquisitionmodality, in accordance with the techniques described hereinabove.Subsequently, locations along the lumen of an endoluminaldata-acquisition device associated with a second data-acquisitionmodality (e.g., OCT) are identified as corresponding to respectiveendoluminal data points of the second data-acquisition modality, inaccordance with the techniques described hereinabove. For example,forward motion of one or both of the endoluminal data-acquisitiondevices may be accounted for in associating the locations of theendoluminal data-acquisition devices with the image frames, inaccordance with techniques described hereinabove. Consequently, the twodata sets are co-registered to one another. For some applications, thetwo endoluminal data sets are displayed overlaid or otherwise mergedwith one another.

Reference is now made to FIG. 5, which is schematic illustration of areference tool 100 having coupled thereto radiopaque markers 102, acharacteristic of the markers varying along a least a portion of thereference tool, in accordance with some applications of the presentinvention. For example, as shown, the separation between adjacentmarkers may vary along at least a portion of the reference tool.Alternatively or additionally, also as shown, shapes of the markers mayvary along at least a portion of the reference tool. For someapplications, the reference tool is a wire (e.g., a guidewire), or asheath that is used to facilitate the insertion of an endoluminal device(e.g., an endoluminal data-acquisition device) into a lumen.

For some applications, reference tool 100 is inserted into a lumen. Anendoluminal device (e.g., an endoluminal data-acquisition device) isinserted into the lumen under extraluminal imaging (e.g., fluoroscopicimaging), the endoluminal device having a radiopaque portion (e.g., aradiopaque marker) associated therewith. For example, the data-acquiringportion of an endoluminal data-acquisition device may be radiopaque,and/or may have radiopaque markers coupled thereto. The location of theendoluminal device within the lumen is determined by determining, viaimage processing, the location of the radiopaque portion that isassociated with the endoluminal device, with reference to the radiopaquemarkers of the reference tool. For some applications, by determining thelocation of the radiopaque portion that is coupled to the endoluminaldevice with reference to the radiopaque markers of the reference tool,errors in the determination of the location of the endoluminal devicewith respect to the lumen (e.g., errors that are caused byforeshortening of the lumen) are reduced, relative to if the system werenot to use the radiopaque markers of the reference tool as referencepoints.

For some applications, the distances between respective pairs of markersthat are adjacent to one another varies along the length of thereference tool, and/or a shape or pattern of the markers varies alongthe length of the reference tool. For some applications, using such areference tool facilitates determining the location of the endoluminaldevice with reference to the radiopaque markers of the reference tool,even if only a portion, and not all, of the markers on the wire arevisible to the extraluminal imaging system. For example, the shapes orpatterns of the markers and/or the distances between respective pairs ofmarkers that are adjacent to one another may vary such that any set ofmarkers (e.g., any pair, or set of three or four of the markers) has aunique appearance. Thus, when the radiopaque portion that is coupled tothe endoluminal device appears in an image in a vicinity of a given setof markers, the location of the device along the lumen with respect tothe reference tool may be determined by the system.

For some applications, reference tool 100 is used together with anendoluminal data-acquisition device in order to facilitate registrationof endoluminal data points that are acquired by the data-acquisitiondevice to an extraluminal image of the lumen, for example, usinggenerally similar techniques to those described herein and/or generallysimilar techniques to those described in US 2012/0004537 and/or WO12/014,212, both of which applications are incorporated herein byreference.

It is noted that although some techniques for co-using extraluminalimages and endoluminal data are described hereinabove primarily withrespect to extraluminal fluoroscopic/angiographic images and endoluminalIVUS images, the scope of the present invention includes applying thetechniques described herein to other forms of extraluminal andendoluminal images and/or data, mutatis mutandis. For example, theextraluminal images may include images generated by fluoroscopy, CT,MRI, ultrasound, PET, SPECT, other extraluminal imaging techniques, orany combination thereof. Endoluminal images may include images generatedby optical coherence tomography (OCT), near-infrared spectroscopy(NIRS), intravascular ultrasound (IVUS), endobronchial ultrasound(EBUS), magnetic resonance (MR), other endoluminal imaging techniques,or any combination thereof. Endoluminal data may include data related topressure (e.g., fractional flow reserve), flow, temperature, electricalactivity, or any combination thereof. Examples of the anatomicalstructure to which the aforementioned co-registration of extraluminaland endoluminal images may be applied include a coronary vessel, acoronary lesion, a vessel, a vascular lesion, a lumen, a luminal lesion,and/or a valve. It is noted that the scope of the present inventionincludes applying the techniques described herein to lumens of asubject's body other than blood vessels (for example, a lumen of thegastrointestinal or respiratory tract).

It will be appreciated by persons skilled in the art that the presentinvention is not limited to what has been particularly shown anddescribed hereinabove. Rather, the scope of the present inventionincludes both combinations and subcombinations of the various featuresdescribed hereinabove, as well as variations and modifications thereofthat are not in the prior art, which would occur to persons skilled inthe art upon reading the foregoing description.

The invention claimed is:
 1. Apparatus comprising: an endoluminaldata-acquisition device configured to acquire a plurality of endoluminaldata points while moving through a lumen of a subject's body; a secondendoluminal device configured to be moved through the lumen; a display;and at least one processor configured to: determine that respectiveendoluminal data points correspond to respective locations along thelumen, generate a stack of the endoluminal data points, in which theendoluminal data points are positioned at locations corresponding torelative locations within the lumen at which the endoluminal data pointswere acquired, while the second endoluminal device is inside the lumen,and while the endoluminal data-acquisition device is in a retrievedstate with respect to the lumen, subsequent to the endoluminaldata-acquisition device having acquired the plurality of endoluminaldata points while moving through the lumen, determine a current locationof the second endoluminal device with respect to the lumen, and drivethe display to display the stack, and to display within the stack animage of the second endoluminal device at a location within the stackcorresponding to the current location of the second endoluminal device.2. The apparatus according to claim 1, wherein the processor isconfigured to drive the display to display the image of the secondendoluminal device within the stack by driving the display to display avirtual representation of the second endoluminal device within thestack.
 3. The apparatus according to claim 1, wherein the processor isconfigured to drive the display to display the image of the secondendoluminal device within the stack by driving the display to display areal image of the second endoluminal device within the stack.
 4. Theapparatus according to claim 1, wherein the second endoluminal devicecomprises a therapeutic endoluminal device.
 5. The apparatus accordingto claim 1, wherein the endoluminal data-acquisition device comprises anendoluminal imaging device that is configured to acquire a plurality ofendoluminal images while the endoluminal imaging device is being movedthrough the lumen, and wherein the processor is configured to generatethe stack by generating an endoluminal image stack.
 6. The apparatusaccording to claim 1, wherein the endoluminal data-acquisition devicecomprises an endoluminal data-acquisition device that is configured toacquire functional data regarding the lumen while the endoluminaldata-acquisition device is being moved through the lumen, and whereinthe processor is configured to generate the stack by generating a stackof functional endoluminal data points.
 7. The apparatus according toclaim 1, wherein the processor is configured to generate the stack ofendoluminal data points by generating a stack of indications of theendoluminal data points, locations of the indications within the stackcorresponding to relative locations within the lumen at which theendoluminal data points were acquired.
 8. The apparatus according toclaim 1, wherein the processor is configured to generate the stack ofthe endoluminal data points, by generating a stack of endoluminal datapoints that includes a gap at a location within the stack correspondingto a location within the lumen at which no endoluminal data points wereacquired.
 9. The apparatus according to claim 1, wherein the processoris configured to generate the stack of the endoluminal data points, bygenerating a stack of endoluminal data points in which duplicateendoluminal data points are removed from the stack at locations withinthe stack corresponding to locations within the lumen at which more thanone endoluminal data point was acquired.
 10. The apparatus according toclaim 1, wherein the processor is configured to generate the stack ofthe endoluminal data points, by generating a stack of endoluminal datapoints, by accounting for non-longitudinal motion undergone by theendoluminal data-acquisition device, while acquiring the plurality ofendoluminal data points, while moving through the lumen.
 11. Theapparatus according to claim 1, wherein the processor is configured togenerate the stack of the endoluminal data points, by generating a stackof endoluminal data points that is calibrated with respect to physicaldimensions of the lumen.
 12. The apparatus according to claim 1, whereinthe processor is configured to generate the stack of the endoluminaldata points, by generating a stack of endoluminal data points, a crosssection of the stack being calibrated with respect to physicaldimensions of the lumen.
 13. A method for use with an endoluminaldata-acquisition device configured to acquire endoluminal data pointswhile moving through a lumen of a subject's body, a second endoluminaldevice, and a display, the method comprising: while the endoluminaldata-acquisition device is being moved through the lumen, acquiring aplurality of endoluminal data points of the lumen using the endoluminaldata-acquisition device; using at least one processor, determining thatrespective endoluminal data points correspond to respective locationsalong the lumen; using the processor, driving the display to display atleast some of the plurality of endoluminal data points in a stack, inwhich the endoluminal data points are positioned at locationscorresponding to relative locations within the lumen at which theendoluminal data points were acquired; using the processor, while thesecond endoluminal device is inside the lumen, and while the endoluminaldata-acquisition device is in a retrieved state with respect to thelumen, subsequent to the endoluminal data-acquisition device havingacquired the plurality of endoluminal data points while moving throughthe lumen, determining a current location of at least a portion of thesecond endoluminal device with respect to the lumen; and in responsethereto, displaying within the stack an image of the second endoluminaldevice at a location within the stack corresponding to the currentlocation of the second endoluminal device.
 14. The method according toclaim 13, wherein displaying the image of the second endoluminal devicewithin the stack comprises displaying a virtual representation of thesecond endoluminal device within the stack.
 15. The method according toclaim 13, wherein displaying the image of the second endoluminal devicewithin the stack comprises displaying a real image of the secondendoluminal device within the stack.
 16. The method according to claim13, wherein the second endoluminal device includes a therapeuticendoluminal device, and wherein determining the current location of atleast a portion of the second endoluminal device with respect to thelumen comprises determining a current location of at least a portion ofthe therapeutic endoluminal device with respect to the lumen.
 17. Themethod according to claim 13, wherein the endoluminal data-acquisitiondevice includes an endoluminal imaging device that is configured toacquire a plurality of endoluminal images while the endoluminal imagingdevice is being moved through the lumen, and wherein driving the displayto display at least some of the plurality of endoluminal data points ina stack comprises driving the display to display an endoluminal imagestack.
 18. The method according to claim 13, wherein the endoluminaldata-acquisition device includes an endoluminal data-acquisition devicethat is configured to acquire functional data regarding the lumen whilethe endoluminal data-acquisition device is being moved through thelumen, and wherein driving the display to display at least some of theplurality of endoluminal data points in a stack comprises driving thedisplay to display a stack of functional endoluminal data points. 19.The method according to claim 13, wherein driving the display to displayat least some of the plurality of endoluminal data points in a stackcomprises driving the display to display a stack of indications of theendoluminal data points, locations of the indications within the stackcorresponding to relative locations within the lumen at which theendoluminal data points were acquired.
 20. The method according to claim13, wherein driving the display to display at least some of theplurality of endoluminal data points in a stack comprises driving thedisplay to display a stack of endoluminal data points that includes agap at a location within the stack corresponding to a location withinthe lumen at which no endoluminal data points were acquired.
 21. Themethod according to claim 13, wherein driving the display to display atleast some of the plurality of endoluminal data points in a stackcomprises driving the display to display a stack of endoluminal datapoints in which duplicate endoluminal data points are removed from thestack at locations within the stack corresponding to locations withinthe lumen at which more than one endoluminal data point was acquired.22. The method according to claim 13, wherein driving the display todisplay at least some of the plurality of endoluminal data points in astack comprises driving the display to display a stack of endoluminaldata points, by accounting for non-longitudinal motion undergone by theendoluminal data-acquisition device, while acquiring the plurality ofendoluminal data points, while moving through the lumen.
 23. The methodaccording to claim 13, wherein driving the display to display at leastsome of the plurality of endoluminal data points in a stack comprisesdriving the display to display a stack of endoluminal data points thatis calibrated with respect to physical dimensions of the lumen.
 24. Themethod according to claim 13, wherein driving the display to display atleast some of the plurality of endoluminal data points in a stackcomprises driving the display to display a stack of endoluminal datapoints, a cross section of the stack being calibrated with respect tophysical dimensions of the lumen.